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ANNOT ATION 


Linear programming provides methods for computing the most 
practical solutions in production, operations, and supply planning 
and in the control of complex processes. The introduction of 
linear programming into practice materially cuts costs and time 
losses. 

The present book presents in detail the mathematical theory of 
linear programming and computational methods yielding an exact 
solution over a finite number of steps. 

The book is intended for engineers, economists, and applied 
mathematicians, It may also be used by university students in the 
mathematics, economics, and engineering economics departments, 


PREFACE 


Planning and control are among the most complex operations in industrial 
management, economic administration, and staff work at various levels. 
The actual procedure of planning and control determines the way to the 
final objective and essentially affects the quality of the decisions. 

For a long time technological control and planning of economic develop- 
ment or strategic operations could be carried out only by individuals with 
Suitable education, experience, and intuition. However, technological 
progress and present-day complexity of the structure of industry, distri- 
bution of labor and large-scale cooperation, all these have made the prob- 
lem of planning and control highly complicated. 

To arrive at a sound decision, an enormous volume of information must 
be collected and processed. Responsible decisions often involve the fate 
of many people and high material cost. Now it is no longer enough to in- 
dicate the way to the final objective. Of all the possible ways it is necessary 
to choose the most economic, which allows for the specific features of 
flow and development of the controlled process and which gives best results 
in the given circumstances, 

Previous methods of decision making in planning and control problems, 
entailing expansion of the administrative body, may eventually lead to total 
loss of control in the present state of technology. 

The advent of high-speed computers created a powerful precedent for 
complete automation of the numerous control problems, However, this 
possibility could be realized only after the development of suitable mathe- 
matical methods reducing the solution of planning and control problems to 
a succession of automatic operations based on the initial input data. 

The new mathematical discipline serving as the theoretical basis of the 
solution of planning and control problems was called mathematical 
programming. 

In stating a mathematical-programming problem, the purpose of the 
control andthe restrictions to be imposed on the choice of the control para- 
meters, as arising from the specific features of the control process, must 
be clearly indicated. In solving the problem, the system of parameters 
ensuring optimal (in the sense defined) quality of the control process within 
the framework of the stated restrictions is chosen. 

The most developed branch of mathematical programming is linear 
programming — theory and methods of solution of conditional extremum 
problems, in which the quality function is a linear function of the control 
parameters, and the restraints are linear equalities or inequalities. 

Works on linear-programming were first published by Soviet scientists 
(/61-—63, 66/). In recent years various collections have been published in 
the Soviet Union (/96-—99/) presenting the reader with original and trans- 
lated works dealing with particular methods and various applications of 
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linear programming. However, all these articles, and even the highly 
rigorous and concise presentation of the principles of linear programming 

in Kantorovich's book /65/ can hardly be used as a guide in the systematic 
study of this mathematical discipline. The short monographs of Barsov /3/, 
Churnes, Cooper, and Henderson /118/, and Wayda /15/ cover only a 
rather narrow field of problems relating to the principles of linear program- 
ming. The works of Gabr /17/, Gerchuk /21/, Kreko /69/, Reinfeld and 
Vogel /88/ are intended for the unqualified reader interested mainly in 
economic applications of linear programming. Mathematical rigor and 
comprehensive analysis are sacrificed here in favor of readability. 

Among the most comprehensive and also sufficiently rigorous modern 
reference books in linear programming we must mention the book /129/ and 
also the translation of the monograph /18/ by Gass. However, the attempt 
to give in one book general and special problems, theory and methods, and 
mathematical principles and applications made it impossible for the writer 
to devote due attention to each matter. 

The aim of the authors of this book is to present the main aspect of 
linear programming — the theory and the finite methods — whichis ofinterest 
to mathematicians, economists, and engineers, and to try, in this way, to 
achieve better mutual understanding of specialists taking part in the state- 
ment and the solution of planning and control problems. This book, in 
considerably greater detail than any other publication known to us, presents 
the general linear programming theory and gives a very thorough descrip- 
tion of the finite methods and the corresponding algorithms. Much attention 
is devoted to various modifications of methods and algorithms which enable 
the structure and the special features of modern computers to be most effi- 
ciently utilized. Geometrical and economic interpretations of problems and 
methods of linear programming dealt with in detail in this book should be 
of considerable help in understanding more thoroughly the theory and also 
in developing an intuitive grasp of the principles of computational methods 
and algorithms. 

The book contains some new results published for the first time, The 
selection and presentation of the material is intended to emphasize that 
linear programming is not an independent discipline, but a branch of 
mathematical programming. In particular, the finite methods of linear 
programming are classified so as to provide an analogous approach to 
methods of solution of nonlinear extremum problems. 

The structure of chapters adopted in this book makes it possible for 
specialists of different mathematical background and with different interests 
to choose a sequence of study which seems best suited for their needs. 

For the reader's convenience, we add an Appendix giving the elements of 
linear algebra and the concepts of many-dimensional spaces, which are 
useful in the mathematical apparatus of linear programming. It should be 
emphasized that to grasp effectively the methods presented in the book 
the reader must have knowledge not only of the computational, but also of 
the theoretical aspects of linear programming. 

In Chapter 1 the basic concepts of linear programming are presented, 
the relative place of this discipline among other branches of mathematical 
programming is indicated, and various methodological problems, which 
are of significance in the statement of problems and assimilation of the 
methods, are discussed, 


In Chapter 2 the relationship between linear programming and the theory 
of convex polyhedral sets is established, The interpretation of the problem 
in terms of tne theory of convex polyhedral sets makes it possible to sub- 
stantiate various important results of linear programming and helps in 
assimilating the geometrical side of the problem and of methods of its solu- 
tion, 

Chapter 3 deals with the most important theoretical question of linear 
programming-— the duality theory. The duality theory plays an essential 
role in the construction of linear-programming methods. In this chapter the 
relationship between the solution of the dual problem and the so-called 
decision multipliers — analogs of Lagrange multipliers — are established, 
The results of this chapter make it possible to devise a uniform approach to 
the different procedures used in the development of linear-programming 
methods. 

Linear-programming methods, like the methods of linear algebra, are 
subdivided into finite and iterative. Previous experience in the solution of 
applied problems shows the finite methods to be, on the whole, more effect- 
ive, Chapters 4 to 8 of the monograph present, in detail, the theory and 
the computational algorithms of the basic finite methods of linear program- 
ming. When describing the methods the authors paid attention both to 
matters of principle, which are of interest in the further development of 
the methods, and to special features of the computational schemes, accord- 
ing to which the method and algorithm for the solution of any particular 
problem are chosen, 

The methods and the algorithms are described in application to the 
general linear-programming problem given in the so-called canonical form 
with the variables bounded on one and two sides, In the concluding chapter 
a procedure is indicated for the construction of computational schemes for 
linear-programming problems written in different forms. This chapter 
also gives some modifications of the finite methods ensuring more effective 
utilization of the working memory of the digital computer. 

The classification of the methods according to the various criteria given 
in Chapter 8 leads to the conclusion that the methods discussed in this mono- 
graph can be regarded as typical representatives of all essentially different 
groups of finite linear-programming methods. 

The reader who is interested only in the application of linear-program- 
ming may pass, after studying Chapter 1 (and §§ 1 and2 of the Appendix), to 
the study of the various methods and computational schemes. On first 
reading one may omit §§ 4-6 and 8 of Chapter 4, § 9 of Chapter 5, §§2 and 
4 of Chapter 6, and § 3 of Chapter 7. 

To grasp the techniques of numerical solution of formalized linear- 
programming problems, it suffices to study §§ 2, 3, 5,6, and 8 of Chapter 5, 
§§ 6 and 8 of Chapter 6, and §§ 5 and 6 of Chapter 7. The information 
presented in these sections together with that given in § 2 of Chapter 8 are 
sufficient for compiling a list of finite-method procedures for the solution 
of linear programming problems on digital computers. 

Readers concerned withthe development of theory andfurther perfection 
of the linear-programming methods should concentrate, besides Chapters 
2 and 3, on those sections of Chapters 4 to 7 which deal with geometrical 
interpretation of the methods, the phenomena of degeneracy and cycling, 
and the specific features of problems with bilateral restraints. 
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The exercises given at the end of each chapter, besides numerical 
examples conducive to better understanding of the material, also include 
questions and problems which, for various reasons could not be included 
in the text. 

The numeration of formulas in the book is autonomous for each section. 
The number of the formula consists of two numerals: the first indicates 
the number of the section, the second the serial number of the formula in 
the section. When reference is made to formulas from other chapters, the 
chapter number is also indicated, 

The authors will be sincerely grateful to those readers who will take the 
trouble of sending in their remarks on the contents of this book. 


October 1961 The Authors 


Chapter 1 
FUNDAMENTAL CONCEPTS OF LINEAR PROGRAMMING 
§ 1. The subject of mathematical programming 


1-1, The development of mathematics at the end of the 19th and the be- 
ginning of the 20th century was shaped mainly by the requirements of phys- 
ical problems. Theoretical physics was responsible for the progress in 
differential geometry, vector and tensor analysis, the theory of differential 
and integral equations, probability theory and mathematical statistics as 
well as several other mathematical disciplines. The fundamental concept 
of the mathematical sciences in that period was continuity. The natural 
sciences and technology did not pose large-scale combinatorial problems to 
mathematicians, and furthermore, the available computational techniques 
were not able to provide effective methods for the solution of more complex 
problems. 

In the last decade, however, various problems emerged in industrial or- 
ganization, economic planning, automation of industry, and weapons control. 
A natural reaction to this trend was the development of cybernetics. Prob- 
lems of power and technology control became as significant as problems of 
power and generation and technical development, It became obvious that 
the economic effect of effective control and planning methods, when applied 
on a large scale and ona Suitably high level, may, in some cases, exceed 
the effect of a substantial increase in power output. New mathematical 
methods were required to analyze industrial processes, the operation of 
the nervous system of a living organism, the interactions between individuals 
and society and the quality of the military sciences, Although continuity, 
as before, is the central concept of contemporary mathematics, growing 
attention is given to discrete combinatorial problems. 

The feasibility of solving numerically laborious mathematical problems 
within a reasonable time is now equally as important as proving difficult 
abstract propositions and deriving elegant formulas, Incidentally, the com- 
plicated modern problems of mathematics are not amenable, asa rule, to 
description in terms of formulas relating the given and the required quanti- 
ties. In those cases where qualitative estimates are insufficient, it is 
better to try and develop algorithms which make it possible to calculate the 
result for each concrete system of initial data. | 

The role of physics as the purveyor of new problems and trends in 
mathematics has, thus, recently been replaced by cybernetics. The emer- 
gence and development of digital computational techniques led, of necessity, 
to the creation of new mathematical methods which, in their turn, extended 
the possibilities of mathematical machines. 


1-2. Computers introduced into industry and control and employed in 
scientific research work open enormous vistas for developing various 
branches of science and for perfecting methods of industrial planning and 
automation. However, the applications of computational techniques cannot 
attain the required standard unless the problems are rigorously formulated 
and the processes described mathematically. The difficulty is often created 
not by problems of computer design, but rather by problems arising from 
the formal description of physical, economical, technical and other pro- 
cesses. Formulation of a problem is a necessary stage preceding its trans- 
lation to the language of computers. 

The general formulation of the problems of industrial control and plan- 
ning can be described as follows. 

The problems of control and planning often reduce to the choice of a 
particular system of parameters and functions. We shall refer to these 
parameters and functions as the control characteristics. To choose any 
particular set of control characteristics, we must clearly establish two 
points. First, we must decide what is a good system, and further what is 
the best system of planning and control. In other words, we should formu- 
late and express in terms of the required characteristics a measure of 
quality, i.e. , find a criterion specifying to what extent the plans being de- 
veloped achieve the initial purpose of the process. Second, we must eluci- 
date the operating conditions of the system and the ensuing restrictions to 
be satisfied by the computational characteristics. 

For the present we are concerned with problems whose control charac- 
teristics are systems of parameters, rather than functions. 

The problems of planning and control can thus be reduced to extremum 
problems of the following type. 

It is required to determine the maximum of the function 


iC nee Premera a (1.1) 
subject to the conditions 
Bi (X,) Xy, --- 5%) =O, i=1, 2,...,m, (1.2) 
x20, j=l, 2,...,t<—n. (1.3) 
The function f(x,,...,x,) is the quality measure ofthe solution. Conditions 


(1.2)-(1.3) are the restraints of the problem. Conditions (1.3), which are 
inherent for numerous problems in which the variables x,— the control 
parameters — physically cannot take on negative values, should be separa- 
ted from restraints of the type (1.2), 

The restrictions imposed onthe problem may also be expressed in terms 
of equalities. In the system (1.2) the equality g,(x,,...,x,)=0is represented 
by the two inequalities 

By (X,, ++. ,%,) =O, 
— By (X,,.+.,%,) 0. 


Extremum problems of the type (1.1)-(1.3) should be considered in the 
analysis of various problems of national-economy control and of industrial 
automation. Economic planning as well as military strategy and logistics 
also give rise to problems of this type. 

The mathematical discipline concerned with problems of type (1.1)=(1.3) 
and with the development of methods of solution under various assumptions 
imposed on the functions f and g, is called mathematical program- 
ming. 


The term mathematical programming is unsatisfactory since program - 
ming generally indicates the compilation of computational routines, i.e., 
programs for computers. Possibly a more suitable term describing the 
problems treated in this independent branch of applied mathematics would 
be mathematical planning. A still better description of the subject 
is methods of solution of conditional extremum problems. 
However, mathematical programming has been universally ac- 
cepted in the literature and we shall therefore use it here. 

The principal feature of mathematical-programming problems, which 
distinguish them from ordinary extremum problems of classical analysis, 
is the appearance of inequalities among the restrictions (1.2) and (1.3). The 
methods of solution of conditional-extremum problems with the aid of the 
so-called Lagrange multipliers do not apply here in their original form. 
The classical methods of differential calculus have been developed in order 
to determine the extremum points inside the domain of definition of the 
function. Lagrange's method could, therefore, be applied directly to the 
solution of problems (1.1)-—(1.3) if it were known in advance which of the 
conditions (1.2), (1.3) reduce to equalities for the values of the arguments 
specified by the solution of the problem. However, generally speaking, no 
such information is available. If we confine the procedure to the techniques 
of classical analysis, we may indicate the following method for solving the 
mathematical -programming problem. 

Let the functions f and g, satisfy all the requirements necessary for 
solving conditional-extremum problems by Lagrange's method. 

We first calculate the absolute extremum (maximum) of the function 
f(%,,..-,%,), neglecting the conditions restricting the range of the variables, 
and then check whether the coordinates of the extremum point also satisfy 
conditions (1.2), (1.3). If these conditions are satisfied, the point is said 
to be the solution of the problem. If at least one of the conditions (1.2), 
(1.3) is not satisfied, Lagrange's method should be used to determine the 
conditional extremum of the function f(x,,...,x,) for one of the conditions. 
Then we check whether the arguments at which the conditional extremum is 
attained satisfy the remaining conditions of the problem. Analogous cal- 
culations are carried out under the assumption that the solution of the 
problem reduces to equalities any other condition among (1.2), .(1.3), any 
pair of conditions, any triad of conditions, etc. In all these cases, when 
the extremum point falls within the domain specified by conditions (1.2), 
(1.3), we calculate the corresponding value of the quality measure of the 
solution, i.e., the function f(x,,...,.,). 

It can be easily observed that this method of solving mathematical-pro- 
gramming problems requires the investigation of an enormous number of 
conditional extremum problems, even if there are relatively few restric- 
tions. Generally, the solution of each problem of this type involves tedious 
calculations. Thus, using the existing computational techniques and com- 
puters it is impractical to solve problems of the type (1.1)-(1.3) by methods 
of classical analysis. Hence the necessity of developing special methods 
of solving these mathematical-programming problems, 

1-3. Mathematical programming is subdivided into several disciplines, 
depending on the properties of the functions f and g,, These have attained 
various levels of development, The simplest branches of mathematical 
programming have, by now, been perfected and methods of solution, includ- 
ing suitable computational techniques, have been developed. In more 


complicated cases we have only rudimentary methods of solution. In some 
classes of mathematical-programming problems only qualitative results 
have been obtained. 

The problems and methods of mathematical programming can be classi- 
fied according to various features. 

Depending on the type of the measure function and of the restrictions, 
mathematical programming is divided into linear and nonlinear pro- 
gramming. In linear-programming problems, f and g; are linear func- 
tions of the variables x, 

Among nonlinear conditional extremum problems there is a special class 
of problems classified as convex programming. In convex-program- 
ming problems, it is necessary to calculate the maximum of a concave func- 
tion on a convex set*, Any local maximum of a concave function defined 
on a convex set is also its absolute maximum on that set. This proposition 
is the basis of all methods for solving convex-programming problems. 

The solution of convex-programming problems is simplified if restric- 
tions (1.2) can be represented as linear equalities or inequalities. 

Among the problems of convex programming, relatively detailed methods 
are available for problems of quadratic programming which re- 
quire the calculation of the vectors X=(x,,x,,...,x,) satisfying linear equali- 
ties and inequalities and maximizing a sum of quadratic and linear forms: 


Qe v0 rad = BY eypeity + D erty 


It is assumed that the quadratic form is nonpositive definite, i.e., Qisa 
concave function. 

It is obvious that in all the preceding formulations the requirement of 
maximization of a function can be replaced by minimization. In this case, 
however, a concave measure function should be replaced by a convex 
measure, 

Nonconvex problems of nonlinear programming have not yet been tho- 
roughly investigated. 

Mathematical programming is divided into branches which deal with op- 
timal methods of planning and control under conditions of either total 
information or uncertainty, depending on whether the initial para- 
meters of the problem are determined quantities or random factors, The 
analysis of some classes of mathematical-programming problems under 
uncertainty can be carried out by applying convex-programming 
techniques. This method is called stochastic programming. 

In planning and control problems, the solution obtained at one stage 
often greatly limits the freedom of choice in subsequent stages. According- 
ly, it is expedient to divide the problems and methods of mathematical 
programming into single-stage and multi-stage, or into static 
and dynamic problems. 

In principle other classifications of the branches of mathematical pro- 
gramming are also possible. Within each of the branches it is possible to 
distinguish, in turn, between classes of problems requiring special methods 
of solution. Thus, for example, a large class of problems is associated 
with the application of certain standards. The required control parameters 
in these problems can assume only a limited number of discrete 


* For the definition of convex and concave functions and of a convex set, see Appendix, § 3, 


values. The study of these problems, which involve certain difficulties, 
gave rise to the so-called all-integer programming. All-integer 
programming is of special interest since numerous nonlinear nonconvex 
problems of mathematical programming can be reduced to linear-program - 
ming problems, subject to the requirement of integer solutions. 

In some cases the initial parameters of the problem may vary within a 
certain interval. The so-called parametric programming is con- 
cerned with the variation of the parameters of the measure function and 
the restraints imposed on the solution. 

Applied problems of mathematical programming with numerous variab- 
les and restraints often lead to memory overflow in modern digital compu- 
ters. It became, thus, necessary to develop a new trend, the so-called 
block programming, which deals with the possibility of obtaining an 
exact or an approximate solution of a bulky problem from solutions of 
several particular problems with fewer variables and restraints. 

The study of the features of the special classes of applied problems led 
to certain interesting qualitative and quantitative results. However, most 
of these results have not yet been unified. 

i-4, Before concluding this section we find it worthwhile commenting on 
the formulation of mathematical-programming problems. 

Mathematical analysis applies not to actual phenomena, but rather to 
certain mathematical models of these phenomena. These abstract models, 
naturally, do not encompass all, but only the most significant aspects of 
the phenomenon in question. The greatest responsibility lies in chosing 
those characteristics which are the most essential for the given problem and 
should, therefore, be included in the formulation of the mathematical model. 

The phenomena being studied are not isolated. They are related to and 
interact with other phenomena of nature and life, which are possibly of no 
interest to the problem in question. When stating a problem, we must de- 
cide which of the relationships are to be neglected and which are to be re- 
placed by certain restraints imposed on the required parameters. The 
thoroughness exercised in this stage will determine the advisability of using 
a complicated mathematical apparatus in the analysis of the problem and 
in the applicability of the solution. 

In the formulation of planning and control problems special attention 
must be paid to the choice of the measure function of the solution for the 
corresponding mathematical-programming problem. The required planning 
and control method must, often, meet highly diverse, and often contradic- 
tory, requirements. Asa rule, there does not exist a program or a con- 
trol method maximizing (or minimizing) all the various criteria. The ana- 
lysis of the results to be achieved by planning will bring out the most im- 
portant index, which should then be optimized, and also yield the admissible 
limits of variation of the other criteria. The parameter being optimized 
is the quality measure. The admissible limits of variation of the other 
characteristics impose additional restraints on the problem. 

An important stage in problem formulation is the choice of the variables 
which are to be taken as control parameters. The variables should be 
chosen so as to ensure the simplest possible form of the restraints and the 
measure function. Successful choice of the control parameters will deter- 
mine the final amount of computational work necessary in the solution stage. 

The problems of mathematical programming and, in particular, linear- 
programming problems to which the applied problems of planning and 


control are reduced, involve, as a rule, numerous variables and restraints. 
The mathematical models, particularly models of phenomena which are 
studied for the first time, often do not reflect all the conditions. This 
limits the range of the problem variables. Some factors and limitations, 
which appear to be natural, are considered as self-evident and are not sta- 
ted explicitly. If the solution of the problem is not attained by purely 
formal methods, reasoning in terms of concrete applications generally re- 
Sults in the following: the problem is subjected, at the solution stage, to 
additional restraints ensuing from its physical interpretation. However, 

an explicit solution of the problem is possible only if there are only a 

few variables and restraints. In practice, the mathematical-programming 
problems encountered in industrial and economic planning or in the control 
of technical and military operations are highly complicated, and can only 
seldom be completely analyzed. In these cases the problem must be care- 
fully formulated and all the essential restraints, even if trivial, should be 
included in the mathematical model. For example, cases are known when 
solutions of least-cost diet problems, ensuring all the required nutrients, 
yielded quite unedible menus. This was due to the omission of edibility 
criteria from the mathematical model. There are sometimes cases when 
an analysis of mathematical-programming problems gives impracticable 
solutions: this may be so because in the formulation of the problem not all 
the limiting factors affecting the choice of the variables have been taken 
into consideration. The existence of additional restraints often emerges 
only after the causes of the impracticability of the solution have been ana- 
lyzed. 

The statement and the solution of applied problems often cannot be ob- 
tained in a single stage. Quantitative analysis of the solution generally 
points to the way for improvement of the model, with the purpose of attain- 
ing better agreement with the actual phenomenon. After the preliminary 
statement of the problem, it is expedient to obtain the formal solution for 
the simplest cases, when the optimal program is also known (exactly or 
approximately) from physical considerations. The analysis of deviations 
of the formal solutions from the expected result makes it possible to intro- 
duce adjustments to the formulation of the problem or, as may indeed occur, 
may modify our previous notions. Comparison of the formal solutions with 
the expected result must, however, be carried out keeping in mind that 
factors which are essential in a certain range of the initial parameters may 
become quite insignificant in another range. 

Allowing for the additional factors emerging in the formulation stages, 
we may have to modify the choice of the control parameters to be deter- 
mined, The most detailed consideration of the restraints imposed on the 
variables will not yield the solution of the problem if the mathematical model 
cannot be dealt with by existing methods. Serious problems of mathema- 
tical programming are, therefore, best formulated by specialists in the 
applied sciences working in cooperation with mathematicians. 


§ 2. The object of linear programming 


2-1. Linear programming is the best developed branch of mathematical 
programming. The scope and principles of solution of linear-programming 


problems have been formulated fairly rigorously. We may Say that at 
present linear programming constitutes an independent branch of applied 
mathematics which has reached a measure of perfection. 

The object of linear programming is to calculate the extremum (maxi- 
mum or minimum) of linear measure functions subject to the condition that 
the variables to be determined satisfy linear equalities or inequalities. 
Numerous planning and control problems of national economy, technology, 
strategy and logistics are stated in terms of linear programming. The ne- 
cessity for immediate applicability of these problems is the reason for the 
considerable effort being made in developing suitable methods of solution. 

The term linear programming is as unsatisfactory as the term 
mathematical programming. Nevertheless, we use this term here 
too since it is universally accepted. 

We should again emphasize that the word ''programming'"' is used here 
in the sense of the development of methods of solution of extremum prob- 
lems, and the adjective ''linear" indicates that the measure function and 
the restraints imposed on the unknown variables are linear functions. 

The general linear-programming problem is stated as follows: 

It is required to determine the maximum ofa linear function of a varia- 
DIOS 4g Wises b5. ke 


L=L(x,,...,%,)=C,%,+6,%,+...+6¢,%, (2.1) 
subject to the following restraints imposed on the variables «x,,...,x,: 


@,,%,+4,.%,+...+6,,%, =), 
be ok Hae Se, Gre ee es (2.2) 
G,,X%,+4,%, +... +2, X, =), 
Brass +4745 Xt ers ® + 4,,,, n*n <= Dress 
ar ee A RR SS we SE Ge Ce SS (2.3) 
@,,%, + a,,%, + 1+» +$2,, x, = 5,, 
x,2=0, j=l, 2,...,¢ (ta). (2.4) 


The linear function (2.1), the quality measure of the variables in ques- 
tion, is generally called the linear form of the problem, and the set 
(x,, X,,---,%,) Satisfying conditions (2.2)-(2.4), isthe domain of defi- 
nition of the problem, or the domain of definition of its linear form. 

Let us now introduce some additional definitions which will be rendered 
more precise in the following in the application to the various mathematical 
forms of the linear-programming problem*, 

The matrix of the coefficients 


ay, ais e< 
A= e e e e e e 
a, ars 229 Grn 


will be called the restraint matrix of the problem. 
The columns of the restraint matrix, i.e., the vectors comprising the 
coefficients of the variables x,, will be called the restraint vectors 


A, = (a,;, Gs) eee, a,,)". 
(T, the transposition symbol, indicates that the vector A,, whose compo- 
nents are arranged inarow, is a column vector.) 


* The reader who is not familiar with the concepts of linear algebra used here will find the necessary expla- 
nations in the Appendix. 


The vector, whose components are the right-hand sides of the problem 
conditions, will be called the constraint vector 
B=(b,, b,, «++, 0,)7. 
Let ; : 
Ay= (Bij App vees Oy)", Ap (Oras pp Braap veer ps)" 


and, correspondingly, 


B’=(0;, 6. 3224 0). B= (Oo Ores: 23s. By)" 
Then 
A,= (Aj, Aj), 
B=(B’, B’), 


and the problem (2.1)-(2.4), can be rewritten in the following compact form: 
Maximize the linear form 


n 
1 ee t= 297; (2.5) 
subject to the conditions 
n o 
Arne (2.6) 
m1 
BAe se, (2.7) 
=t 
x,20, j=l, 2, ..., ta. (2.8) 


A vector X=(x,, x,,...,%,), satisfying conditions (2.2)-(2.4), or equiva- 
lently (2.6)-(2.8), willbecalled a feasible program ofthe problem. 

The program X*=(x‘, ..., x.) maximizing the linear form (2.1) is called 
the optimal program, orthe solution of the problem. 

The terms program and optimal program originating in eco- 
nomic applications are also retained in the general linear-programming 
problem. 

In what follows we shall operate with the matrix form of linear-program - 
ming problems. 

Let us introduce the matrices 


A’ =(Aj, Ag, «+e) An)s 
A” =(A,, Ay, owes An) 


and the vectors 
C=(C,, €,, ose) Cy) 


De ee ees 2B 
X” = (Xa) pony x,)s 
X=(X', xX’)? 


The general linear-programming problem in these new notations is 
formulated as follows: 
Calculate the vector X=(X’, X")' maximizing the linear form 


L(X)=CX (2.9) 
subject to the conditions 
A'X=B’, (2.10) 
A’X <= B’, (2.11) 
x > 0. (2,12) 


The methods and algorithms of linear programming are discussed in 
what follows mainly for problems requiring maximization of the linear form. 
This assumption obviously does not detract from the generality of the dis- 
cussion. The problem involving minimization of the linear form reduces 
to the maximization problem when the signs of all the coefficients c, are 
reversed. 

In the analysis of the conditions (2.2)-(2.4), three cases may arise: 

(a) conditions (2.2)-(2.4) are inconsistent, i.e. , there does not exist a 
set of numbers x,, x,, ..., *,, satisfying all the conditions of the problem; 

(b) conditions (2.2)-(2.4) are consistent, but the domain defined by these 
conditions is not bounded, i.e., there exist sets of numbers «x,, %,, .... X, 
satisfying conditions (2.2)~(2.4) and containing individual variables of ar- 
bitrarily large values; 

(c) conditions (2.2)-(2.4) are consistent and the domain defined by these 
conditions is bounded. 

Let us now introduce the concept of solvability of a problem which willbe 
useful in the subsequent discussion. 

A linear-programming problem is said to be solvable if there exists 
a set of numbers (x,,x,,...,%,) (*,;<00o), which satisfy all the restraints (2.2)- 
(2.4) and, depending on the problem requirements, maximize or minimize 
the linear form (2.1 ). 

Unsolvability of a problem may be due to either inconsistency of the prob- 
lem conditions (case (a)), or to the lack of boundedness of the linear form in 
its domain of definition (case (b)). Observe that case (b) does not necessarily 
result in an unsolvable problem. A linear form may be bounded even in 
an unbounded domain. 

2-2. In the following (Chapter 2) we shall show that the domain of defi- 
nition of the linear form of a linear-programming problem is a convex poly- 
hedral set in n-dimensional space of the variables x,, and the extremum 
of the linear form is attained at the vertices, which are finite in number. 
The vertex coordinates of the polyhedral set of problem conditions satisfy 
the equalities corresponding to linearly independent restraints of the type 
(2.3)-—(2.4). Therefore, the theoretical method of solution of the mathema- 
tical-programming problem outlined in 1-2 is considerably simplified in 
the linear case. There is no need to look for extrema for 1, 2,...,2—1 
constraining equalities. The coordinates of the extremum point in a linear- 
programming problem always satisfy n independent constraining equalities. 
However, even in linear-programming problems the theoretically feasible 
method is impracticable. Even for comparatively small a,r and ¢ (cases 
corresponding to relatively simple applied problems), there are several 
billion vertices of the corresponding polyhedral set. This means that an 
unordered processing of the vertices, with the purpose of finding the point 
at which the linear form is maximized or minimized, is a problem which 
cannot be tackled even on the fastest modern computer, 

To get an idea of the laboriousness of the computations entailed in ran- 
dom processing of the vertices, consider one of the classical problems of 
linear programming — the choice problem — for which we can easily calcu- 
late the number of vertices of the corresponding polyhedral set. In the 
choice problem we are given a tableau of a rows and acolumns, It is re- 
quired to choose one element from each row and each column so that their 
sum is a minimum. This problem, which has diverse practical applications, 


is a linear-programming problem, The number of vertices of the corres- 
ponding polyhedral set (in the choice problem this set is bounded and is 
consequently a polyhedron) equals a: 

The direct solution of the choice problem thus involves the comparison of 
of a! quantities. To calculate the value of the linear form at each of the 
polyhedron vertices, we should carry out 2a summations. For a>15 all the 
operations required to solve the problem cannot be carried out within a 
reasonable time either on existing or on projected computers. 

Even using Stirling's formula 


niee( 2)" V Onn, 


for n=20, the number of polyhedron vertices, al, is greater than 2- 1018, 

A computer performing 10 million operations per second (there are still no 
such computers available) would require 5000 years to process the poly- 
hedron vertices of this relatively simple problem. 

For ana=30, the number of vertices of the domain of definition of the choice 
problem is greater than 1031. It seems that it would be easier to count the 
grains of sand on the earth than to process the polyhedron vertices of this 
problem. 

In practice solutions are required for problems similar to the choice 
problem with a considerably larger than 20 and 30. The preceding examples 
clearly show that special methods must be developed for the solution of 
linear-programming problems. 

As we shall see in the following, the various methods available for solv- 
ing linear-programming problems amount to ordering the processing of the 
vertices of the polyhedral sets defined by the constraints of the given prob- 
lem, or some other problem associated with the original one. Each method 
of linear programming has a characteristic vertex-optimality criterion. 
The optimality criterion makes it possible to establish whether the extre- 
mum of the linear form is attained at the particular vertex, without com- 
paring the values of L at all the vertices of the polyhedral set. The opti- 
mality criteria result from the property that in linear-programming prob- 
lems every local extremum is also an absolute extremum. 

If the vertex considered does not correspond to the extremum of the 
linear form, the next vertex should be taken. Methods of linear program - 
ming direct this processing in such a way that each successive vertex is 
closer to the extremum than the preceding one. Methods of linear pro- 
gramming also make it possible to establish unsolvability of a problem, if 
it is in fact unsolvable. 

Interestingly enough, ordered processing of vertices makes it possible 
to solve the choice problem after about zn’ elementary operations. Exist- 
ing computers then require less than a minute for solving the choice prob- 
lem for a=20 and n= 30. 


§ 3. Linear-programming problems 
In this section we briefly consider some applied problems falling within 


the scope of the model (2.1 )-—(2.4). 
In 3-1, wherethe allocation problem is considered, we outline 
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the characteristic cases in which the solution of the linear-programming 
problem does not require special techniques and cumbersome computations. 

In 3-2 we discuss an important particular linear-programming problem, 
the so-called transportation problem. The transportation problem 
is a natural extension of the allocation problem for a single origin anda 
homogeneous commodity. 

In 3-3 we formulate the optimal weapons problem. Inthis case, 
translation of the measure function and the restraints into formal language 
immediately yields a problem of the type (2.1 )--(2.4). Unlike this problem, 
the multi-component equipment problem, considered in 3-4, 
reduces to a linear-programming problem only after certain (although 
simple) artificial manipulations. 

3-1. We consider the problem of optimal allocation of a homogeneous 
commodity to an industrial center, the commodity being, e.g. , potatoes or 
coal. The commodity in question can be supplied from a origins. 

Let x, be the amount delivered to the destination from the i-th origin, 
and c, the overall cost of producing and shipping a unit amount from the i-th 
origin. Then the cost of the product shipped from the i-th origin to the 
destination is c;x;, and the cost of the product delivered at the destination 
from all the production origins is 


L=c,x,+ex,+...+6¢,%,. (3.1) 


We must allocate the supply, i.e., choose the x; in such a way as to 
ensure minimum delivery cost of the product. The following conditions 
must be taken into consideration in the process: the actual requirement, 
specified by 6, should be satisfied exactly. Hence, the required variables 
should satisfy the condition 


X,+%, +... +4, =. (3.2) 


The production of the commodity at the i-th origin is limited by 5,, and 
the carrying capacity of the shipping routes, which can be chartered for 
shipping the product from the i-th origin, is limited by d,. 

Let fp, be the smaller of the two numbers 5b, and d;, 


6,;= min (6; d,), i= 1, 2, eco; A. 
The required variables x, should thus satisfy the restraints 
x; <= By, f=], 2, 00 ey AL (3.3) 


Since a negative shipment has no valid interpretation for the problem as 
stated, we restrict the x,: 


x, => 0, i=1, 2, ose, A (3.4) 


If the commodity from the i-th origin can be sent by two different types 
of transport, the i-th origin should be considered as two origins with dif- 
ferent production costs. 

We have thus obtained a linear-programming problem. It is required 
to minimize the linear form L (3.1), subject to the linear restraints (3.2)- 
(3.4). 

This problem has a characteristic property which makes it possible to 
compute easily the values of *; for which the cost of the commodity at the 
destination is minimized. 

It is obviously advisable to receive as much of the commodity as possible 
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from those origins for which the c, (the cost of production and shipment per 
unit commodity) are small. Let us renumber the production origins in 
order of increasing c¢;, We now have 


CF. 6,1... Oy. 


By assumption, the destination may receive from the first origin (the one 
supplying the cheapest product) not more than f, units of the commodity. 
If the total requirement of the destination, measured by b, does not exceed 
the resources of the first origin, i.e., if 5<B6,, it would be best to satisfy 
the entire requirement from the surpluses of the first origin. In this case 


X,=b, x,=x,=...=%,=0. 


If, however, b>6,, it would be expedient to ''ship most at the least cost", 
i.e., to take x«,=6,, satisfying the remaining part of the requirements 
(5—B,) from the remaining origins. 

We have here obtained an allocation problem analogous to the preceding 
one, the difference being that the requirements of the destination are speci- 
fied by 6—f, and the number of production origins is now a—1. Here again 
two cases are possible. In the first case(b—BJ<f,. Then the optimal al- 
location system is specified by the equalities 


x,=b—8,, x, =x*,=...=2%,=0- 


(x, = 6, has been previously determined). 

In the second case, where (b—f,) >6,, it is expedient to take x,= 6, and 
to proceed with the problem entailing fewer origins (a—2) and smaller re- 
quirements (6 —6,—f,). 

The preceding reasoning leads to the following procedure for solving the 
problem. 

From 6 we successively subtract the numbers 6,, B,, ... Two cases are 
possible: 


a) 6—f,—B,—...—B,>09; 
b) b-—f,—B,—...—B, <9. 


In the first case the requirements of the destination cannot be completely 
satisfied. The demand exceeds the overall supply potential of all the origins. 

In the second case, the demand can be completely satisfied. 

Let us determine the index ek from the condition 


b—B,—...—B, 0, 
b—-B, — <1 —B,—By4,<0. 
The optimum allocation system is then determined by: 
x, =6,, x,=B,, <se4 x, =,’ 
Xp, =5—B, —B,—. ee —B,' Xpey ee =X, = 0. 


Thus, in the optimal allocation program the first k& origins are relieved 
of all their surpluses, and the subsequent n—(k+1) origins do not enter the 
program at all. 

Observe that we obtained the solution of this problem by elementary 
reasoning. Unfortunately, this is not the rule, but rather the exception in 
linear-programming problems. The simplicity of the solution arises from 
the fact that the problem conditions contain only a single restraint imposed 
on all the variables. All the other conditions only restrict the range of 
each of the variables separately, 
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Let us now make the allocation problem for a single destination and a 
homogeneous commodity more complicated by introducing an additional 
restraint. 

For example, let the loading time of the conveyance intended for ship- 
ping the commodity be limited and, moreover, let the mechanization of the 
loading operations be different in different origins. Let a, be the time re- 
quired at the i-th origin for loading one transport unit and let T be the 
limit imposed on the overall loading time in the network. It is natural to 
assume that the number of transport units y,, required for shipping the 
commodity from the i-th origin is proportional to the bulk of the commodity 
shipped. The idling time of the transport at the i-th origin is given by 
a,y;= a,x, and the total idling time of the network by a(a,x,+a,x*,+...+4,x,). 

The allocation problem is now formulated as follows. Minimize the li- 
near form (3.1) subject to (3.2)-(3.4) and satisfying the additional restraint 


a (a,x, +a,x%,+...+4,%,)< T. 


Similar considerations as in the above will not provide a solution here. 
In this case and, moreover, in those cases where new restraints must be 
taken into consideration, allocation planning requires special methods which 
are the subject matter of linear programming. 

Consider, for instance, the allocation problem for an inhomogeneous 
commodity, i.e. , vegetables or fuel. In this case we can indicate the con- 
ditions of interchangeability, the minimum and the limiting requirements 
of each individual product, e.g., separately for potatoes and cabbage, or 
coal and oil. Interchangeability should be characterized by a coefficient 
indicating how many units of one commodity are equivalent to one unit of 
the other. The concept of equivalence can be defined differently in different 
problems. In our problem, we may assess the interchangeability of the 
commodities from, for example, the calory content of the vegetables or 
the fuel. 

Retaining the notations adopted in the previous case (homogeneous com- 
modity) we now formulate the problem mathematically. 

Minimize the linear form 


Lael eel cl) cc. ec) xf? 
subject to the conditions 
binin <SX1) fos Xn) <a, 
binkn <p? +... + xn’ << Omaxs 
Om xf <p", 
0< x= pt", 
Ban) ee ban) te (mt + oe. ob xn?) =p. 
The superscript enumerates the commodity. If some of the a origins 
produce only one of the commodities being considered, the corresponding 
values of xf!are taken as zero from the outset. The last equality among 
the restraints is the interchangeability condition. It may be interpreted, 
for example, as follows: yp, (x!) +...-+.f) is the number of calories released 
when x,+...+., tons of coal are burnt, p,(x®+...+xf) is the quantity of 


heat released by oil, and uw is the required quantity of heat. Finally, there 
is no difficulty in generalizing the statement of the problem for the case of 
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allocating an arbitrary amount of inhomogeneous commodities to several 
destinations. 

3-2. The allocation of a homogeneous product from m origins to n des- 
tinations, which is the subject of the so-called transportation prob- 
lem, constitutes one of the first applications of linear programming. 
Numerous economic and military problems can be reduced to the formal 
outline of the transportation problem. The essence of the transportation 
problem is as follows. 

There are m origins of a homogeneous product A,, A,, ..., A,. The pro- 
duction capacity of the origin A, is a; units of the commodity. The entire 
supply of the commodity is consumed in a consumption centers B,, B,, ..., B,. 
The consumption capacity of the destination 8, is 6, units of the commodity. 

It is necessary to plan the supply to the destinations B, (j=1,2...., a), 
from the origins A; (i=1, 2,..., m), so that the total transportation costs 
are minimum. It is assumed that shipment can be arranged from any origin 
to any destination, the shipment charges being proportional tothe amount 
shipped. The shipment schedule should fill the requirements of all the con- 
sumption centers without leading to overstocking of the production sources. 
Production and consumption are obviously assumed to balance, i.e., the 
total production capacity equals the total consumption capacity. 

The formal description of the problem requires the following notations. 
Let x,, be the number of commodity units to be shipped from the i-th origin 
to the j-th destination, and c,, the costs incurred in the shipment of one 
unit of the commodity from 4; to B,. 

In the transportation problem it is required to find such values of x;, 
which minimize the total shipment costs 


m nr 


L= DD “iri 
f=1f=1 
subject to the conditions 
nr 
Pati i= 1, 2, asey M,7 (3.5) 
=1 
Pa kis= bp {= 2, eoeey n, (3.6) 
- x; = 0. (3.7) 
Conditions (3.5) indicate that from the origin A; (i=1, 2, ..., m), a; units 


of the commodity, i.e., the entire production capacity of A;, are shipped 
to all the consumption centers. 

Conditions (3.6) indicate that the consumption center B, receives from 
all the production sources 6, units of the commodity, i.e. , an amount equal 
to the consumption capacity of B,. 

Inequalities (3.7) indicate that shipments are arranged only from the 
production sources to the consumption centers; there are no return ship- 
ments. 

This statement implies the equality of production and consumption. Sum - 
ming the equalities (3.5) over i and (3.6) over j, we obtain 


m a b 
=z I 
The transportation problem is one of the most important problems of 
linear programming. The specific conditions of the transportation problem 
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made it possible to simplify essentially the general methods of the solution 
of linear extremum problems as applied in this case. 

3-3. The problem of optimal weapon systems given here is a slight modi- 
fication of the discussion presented in /80/. 

Weapons of various classes can be applied to achieve the same purpose, 
e.g. , to destroy the same objects. It is required to determine in what 
amounts should weapons of different classes be produced to ensure the maxi- 
mum effectiveness of the weapons system for given resources and limited 
funds. From this statement of the problem we consider, for instance, 
the allocation of funds between production orders placed for strategic air- 
craft of various types, missiles, and nonguided strategic rockets. The 
same problem can be easily formulated in terms of the optimal antitank or 
air-defence weapons system. 

One of the principal assumptions in this problem is that each weapon is 
restricted to a definite group of objectives. 

Let x, be the number of weapons of type j, and p, be the average number 
of targets hit by a single weapon of type j. From our assumption, the 
quality measure of the weapons system — the mathematical expectation of 
the number of eliminated objectives — can be written in the form 


M = p,X, +P. X, +10. +Pp% ne (3.8) 


Let the cost of a single weapon of type j be r,, Then, the restriction 
imposed on the number of components in the weapons system by the assigned 
funds, 6, can be represented by the inequality 


PX, +17,%, +... +17,%, =). (3.9) 


A weapon is a multi-component piece of equipment whose production is 
limited not only by capital investments, but also by the necessity of employ- 
ing special apparatus and materials supplies of which are limited. 
The demand of various factors (raw materials, machine shop, man- 
hours, etc.) for weapons of different types is different. Let a,, be the num- 
ber of units of the factor i required for the production of one unit weapon of 
type /, and 6; the restraint imposed on this factor. Then, the additional 
conditions to be imposed on the choice of x, are written in the form 
@,,X, $a,,%, +... +4,,%, ),, 
iG Ms -B, eg > ee sf ee, So wt (3.10) 
Q,,%, + G%, +... $2;,,%, Oy. 

Here s is the number of limiting factors. 

Beside the preceding conditions, weapons production is limited by storage 
and maintenance factors, availability and training of qualified service per- 
sonnel, etc. Some of the required variables (let these be the first k& of the 
system), are therefore additionally restrained by conditions of the form 


x,<=aa, j=l, 2, ..., kaa, (3.11) 


Finally, all the variables (the number of weapons of each given type) 
should obviously be nonnegative, i.e., 


x20, f=1,2,..., 4. (3.12) 


Generally speaking, the physical interpretation of the variables imposes 
another condition, namely that the x, should be integers. Yet departure 
from this requirement in problems involving fairly large x, does not lead, 
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as a rule, to essential deviations from the optimum. With fractional Xs, 
the solution should be considered as the closest integer to x, 

We observe that the choice of the optimal weapons system requires maxi- 
mization of the linear form (3.8), subject to the linear conditions (3.9)- 
(3.12). This is obviously a linear-programming problem. 

If the choice of the optimal weapons system were restrained only by con- 
ditions (3.9), (3.11) and (3.12), the solution of the problem could be derived 
from the simple considerations shown in 3-2. In the general case, however, 
more complicated methods are needed, 

3-4. Consider the multi-component equipment problem, con- 
cerned with planning the production of pieces of equipment comprising a 
elements. For example, this may be a system consisting of a instruments, 
or an instrument made up of a components, etc. 

Orders may be placed with m distinct enterprises possessing different 
machine-tool resources or different potentials. It is necessary to deter- 
mine the distribution of orders between the enterprises which would ensure 
the production of the maximum number of composite systems within a 
scheduled period. Analogous problems are considered when the production 
of a single instrument is planned, with the various components manufactured 
on different machines of the enterprise or shop. Optimal occupancy of the 
equipment should ensure the maximum release of instruments. 

Let unit time be taken as the period for which the entire working pro- 
gram is scheduled. Let the i-th enterprise (the i-th machine) be capable 
of producing a;, elements of equipment of type j/. Assume that all a,,>0, 
i.e., assume that each component can be manufactured on any of the ma- 
chines. Let x,;; be the time during which the i-th contractor (enterprise or 
machine) is occupied with manufacturing elements of type j. The number 
of components of type / released by all the contractors in unit time is 


A, Xp Ay X gz oe Tg Xmp j=1, 2, econ g A (3.13) 


Only assembled products can be delivered, The number of pieces of 
equipment which can be assembled from the manufactured components is 
limited by the class of elements of which fewest are available. In other 
words, the number of assembled units L, produced in unit time equals the 
least of the elements of (3.13) calculated for various /: 


L=min (a, jx.) +4,;%y,+ -.- + 2m Xml: 


The optimality criterion for order placement and equipment occupancy 
is that the time intervals x, be so chosen that each enterprise (each ma- 
chine) has no idle time and that L be maximized. In formal language this 
means that L must be maximized subject to the conditions 


Xi $¢ Xi... +X, = 1, \ 


x1, = 0, i=1, 2. ees y m, j= 1, 2, ene y a. (3.14) 


The first group of conditions requires that the sum of the times taken 
by each contractor in manufacturing the assembly elements be equal to unity, 
i.e. , the time to which the entire program has been scheduled. This indi- 
cates that the enterprises (machines) should have no idling time. 

The second group of conditions indicates that the occupancy time of each 
contractor with each element cannot be negative. 

The problem of maximizing the minimum at which we have arrived is 
easily reduced to a simple linear-programming problem. Indeed, we 
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readily observe that the maximum number of assembled units can be re- 
leased only if the occupancy of the contractors ensures that the various com - 
ponents are all produced in equal numbers, i.e., the solution of the prob- 
lem must satisfy the conditions 


Indeed, had one of these sums been greater than all the others, with 
a,,>0 we could have increased the minimum sum and thus increased the 
number of assembled units by slightly decreasing the excessive components, 
and all this without violating (3.14). This, however, would indicate that it 
is not the optimal system of order placements which has been chosen. 

Hence, the preceding problem may also be formulated as follows: 

Maximize the linear form 


L= a, Xp Og %yqy eee FAQ xX my 


subject to a conditions 
Nip 2% —, tee + Dig my = Fy aX qq + Fy X og oe Figg Xing = 
= a,; Xia +r Fae%es T vee Qns* mg = 


=X in are 46 +o Haan X ans 
x, + 4ig+ aan +44, = 
XX t--- +4, = 


ee e# oOo 8 ° es e e & e e e s 


Xone Xango et X eg, = 1, 
x = 0, fae), 2,250: Ry- Joly Qe Haag Me 


We have thus obtained a linear-programming problem. 


§ 4. Brief historical survey 


Mathematical programming developed out of practical needs some twenty years ago. 

The most developed and comprehensive branch of mathematical programming is linear programming. 
Works dealing with particular problems of linear programming date back to the beginning of the thirties. In 
1931, for example, in Hungary, Egervary /123/ published a paper on the assignment problem, a particular 
case of the transportation problem, It is noteworthy that later the results of this paper were used by several 
authors to develop a highly effective method for the solution of the transportation method (see /70, 71, 111, 
112/), In the literature this method is known as the Hungarian method. 

The term linear programming was coined in America in the mid-forties (the first American paper 
on a particular problem of linear programming was published in 1941 /116/). In the Soviet Union, studies 
in this field began a little earlier. By the end of the thirties several essential results in linear programming 
were established by L. V. Kantorovich /61, 62, 66/*. In particular, in /61/ the powerful method of decision 
multipliers, used in particular when dealing with dual problems, was advanced and applied to some particular 
problems, 

Decision multipliers are also very useful in formulating optimality tests (criteria) of linear problems, 
The set of algorithms entailing decision multipliers is sometimes referred to as the method of decision 
multipliers, Paper /61/ deals with one of the forms of the method of decision multipliers involving mo- 
tion over the boundary of the cone of the given problem (the method of successive residue con- 
traction), A geometrica] interpretation of linear-programming problems is also given. A geometrical 
description of the method of residue contraction was given by G. Sh, Rubinshtein /94, 95/, 

In 1956 Dantzig, Ford, and Fulkerson /52/, proceeding from the aforementioned Hungarian method, 
developed a general linear- programming method which differs from the method proposed by Kantorovich 


* Paper /66/, published in 1949, was completed by L. V. Kantorovich and M. K, Gavurin in 1940, Publica- 
tion was delayed because of the war. Paper /63/ published in 1942 contains a reference to this work. 


17 


in /61/ in insignificant details only, In a joint work Kantorovich and Gavurin /66/ proposed, in application 
to the transportation problem, another form of the method of decision multipliers involving motion outside 
the problem cone (the potential method). 

The first American work on general problems of linear programming /40/ was published by Dantzig in 
1949, It presents the main concepts of the simplex method(the first algorithm) in application 
to nondegenerate linear-programming problems. A method similar to the simplex was proposed by S, I. 
Zukhovitskii in application to problems dealing with the best Chebyshev approximation (see /58, 59, 60/). 

Chames in /117/ gives a procedure for application of the simplex method in degenerate cases (see also 
Dantzig, Orden, and Wolfe /50/), 

The simplex method, its computational procedures, and various revisions were further considered by 
Dantzig and other authors (/41, 50, 51, 78, 85, 14/). 

In 1954 Lemke /73/ proposed another general linear-programming method —the dual simplex 
method, In Lemke‘'s method the solution of the dual problem by the simplex method is presented in terms 
of the primal problems. 

Similar to Lemke's method is the method of leading variables proposed by Beale in /5/. Other 
available methods of linear programming include the gradient method /124/ and Frisch's method of 
logarithmic potential] /113/, both closely related, the double description method /83/, 
Motzkin's relaxation method /82/, Tompkin's projection method /105/, and others, These 
methods are less effective than the three above mentioned genera! linear-programming methods, and their 
modifica tions, 

Another group of methods comprises the so-called iterative methods adapted from numerical 
methods of solution of rectangular games, The equivalence of the programming problem and the game prob- 
lem proved in /43/ makes it possible to solve linear-programming problems by Brown's method /9, 91/, 
Neumann's method /84/, and the method of differential equations proposed by Brown 
and Neumann in /10/, The iterative method developed by Bulavskii /11/ is not quite to the point. 

From experience acquired till now in the solution of linear-programming problems, it seems that the 
iterative method is inferior to other methods, 

Qualitative results and the discussion of the relationship between linear programming and the theory of 
games can be found in /25, 38, 43, 44, 49, 104, 107/, 

Parallel with general methods of linear programming, methods have been developed for the solution of 
certain particular problems, such as the transportation problem, the assignment problem, the traveling- 
salesman problem, the timetable problem, the interindustry problem, etc. The most important results on 
methods of solution of the transportation problem and its various generalizations were published in /1, 22, 31, 
42, 66, 76, 132/. 

Of some interest in practical applications is the generalization of general and particular linear-program- 
ming methods to problems whose variables are restrained on two sides and whose functions are convex and 
piece-wise linear /26, 119, 130/. The computational procedures developed for these problems are very close 
to the algorithms of solution of the corresponding linear problems restrained on one side, 

In recent years several works have been published on linear-programming problems where physical inter- 
pretation of the variables shows that they should be integers, There is a tendency to collect such problems into 
an independent branch of mathematical programming, the so-called integer programming. The main 
results in this field were published in /32, 33, 34, 35, 46, 79/. 

Practical linear-programming problems contain a great many variables and restraints, Only modern com- 
puters can cope with the laborious computational! work involved in the solution of these problems, Methods 
and techniques of computations on digital computers are discussed in /56/ and /37/, The experience ac- 
quired in solving linear-programming problems on analog installations is described in /122/, 

Owing to the limited operative memory of modern computers there arose a need for the so-called 
block programming. The reader is referred to /45/ and /48/ as the main sources of this field, 

In the formulation of planning and control problems we often lack the detailed information necessary for 
unique determination of the functional parameters and the restraints. There are three possible ways proposed 
for overcoming the difficulties involved, First, studying the effect of variations of the linear-form coeffi- 
cients and of the restraints on program optimality /81/, Second, works on the so-called parametric program- 
ming /18, 19/ propose methods which, without substantially increasing the volume of computations, make it 
possible to give solutions for an entire range of variation of one or several parameters on which the linear- 
form coefficients and the restraints depend in a known manner. The third and last way defines a new branch 
of mathematical programming — the so-called stochastic programming or programming under 
uncertainty which deals with the development of effective methods of planning for given statistical 
characteristics of the problem conditions /47, 75, 87/. 

Linear programming is a branch of applied mathematics. However, works on continuous analogs of 
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linear programming /53, 68, 85/ make it possible to employ the concepts of linear programming in various 
branches of mathematics (theory of best approximations, theory of moments, etc,). The application of 
duality to the theory of best approximation is discussed in /27, 28, 29, 30, 114/. 

Soviet researchers have acquired considerable experience in application of linear programming to the 
solution of practical problems /55, 67, 99, 100, 101, 106/. Kantorovich's book /65/ and collections /97/ and 
/98/ edited by V.S, Nemchinov greatly promoted the application of linear programming in economic re- 
search, A large number of different applications of linear programming to industrial and economic planning 
problems was presented at the conference on application of mathematics to economics, held in Moscow in 
April 1960 /106/. Outside the USSR linear programming has found wide military applications /77, 80, 131/ 
and economic applications, in particular in questions of shop, plant, intracompany, and trade planning, A 
survey of applications of linear programming in the USA is given by Gass /18/. This book also gives a de- 
tailed list of references on applications of linear programming to economic and military problems, 

Much less developed are the methods of analysis of nonlinear problems of mathematical programming. 

Kuhn and Tucker /72/ were the first to deal with nonlinear programming in their extension of Lagrange 
multipliers to nonlinear extremum problems with inequality restraints, Qualitative aspects of nonlinear pro- 
gramming are also discussed in /102, 103, 108, 127/. 

Numerical methods of nonlinear programming were mainly developed to include local extrema. These 
methods are clearly of much interest mostly in problems where it is known a priori that the local and the ab- 
solute extrema coincide. 

Convex programming problems (i.e. , problems with a convex functional whose restraints define a con- 
vex set in the space of variables) constitute the largest class of conditional-maximum problems with this 
property. These problems can be approximated with any degree of accuracy by linear problems, General 
methods of convex programming are discussed in /7, 57, 93, 109, 125, 126/. Problems of quadratic program- 
ming maximizing a sum of nonpositive definite quadratic and linear forms subject to linear equality and in- 
equality restraints have been developed to a greater extent than all other problems of convex programming 
/8, 16, 115/. There are also several works dealing with methods of solution of special problems of convex 
programming. Among these problems we have, for example, the transportation post with nonlinear depend- 
ence of shipping cost on shipment volume /4, 20/, Convex-programming algorithms are essentially simpli- 
fied if the problem functional is a sum of separable convex functionals /26, 119/. 

In the solution of mathematical-programming problems with many local extrema considerable difficul- 
ties arise. Until now there are no sufficiently effective methods of solution of these methods, Certain pro- 
gress in this field is due to the so-called trench method proposed by I. M. Gel'fand and M. L. Tsetlin /20/. 
There are also suggestions to apply Monte-Carlo methods to the solution of mathematical-programming prob- 
lems with many local extrema. 

Nonlinear problems with relatively few variables and restraints can be investigated by means of electrical 
analogs /54/. 

Mathematical programming is a rapidly developing new science, Results, methods, and algorithms of 
mathematical programming often appear in various periodicals, New directions in this science are often 
mentioned not in mathematical works, but in publications on applied subjects, This historical survey, there- 
fore, is not comprehensive. The reader wishing to become better acquainted with non-Soviet sources on 
mathematical programming is referred to Rohde's bibliography /92/ and also to annotated bibliographies in 
/90/ and /12/, A fairly comprehensive bibliography will be found in /96/. 


§ 5. Canonical form of linear-programming problems 


5-1. All the examples considered in §3 fall under the general heading of 
linear-programming problems. However, the linear form and especially 
the restraints in the various problems differ essentially. Numerous examp- 
les can be given to show this diversity in the form of linear-programming 
problems. In some cases the required variables depend on one subscript, 
in others, on two. In some problems the restraints are specified by equa- 
lities, in others, by inequalities. Certain applied problems even lead to 
mixed restraints, some of which are linear equations, and some, linear 
inequalities. Not in all problems is nonnegativity of all the variables spe- 
cified. This diversity of restraints requires that special methods be 
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developed for solving distinct classes of problems. Thus, the study of the 
general properties of linear programming and the development of general 
methods and computational algorithms becomes more difficult. It is, there- 
fore, natural to consider the possibility of reducing any linear-programming 
problem to some general form. 

We shall say that a linear-programming problem is given in the canon- 
ical form if it is stated as follows, 

Maximize (minimize) the linear form 


L=C.x, +... +¢2%- 


subject to the conditions 


e 0@© © © @ &© 8© © 8 8 © 8 86 @& 


x,220, ..., Xo 0. 
The linear-programming problem, written in the general form (2.1 )-(2.4), 
can be reduced to canonical form. 
We introduce in (2.1)-—(2.4) the additional nonnegative variables 


Kneis Sagar ces Xn+(r-1) 
x,>0 for i=n+1, ..., a+(r—). 
The restraints (2.2), (2.3) are then equivalent to the following: 
@,,%,+ +O, X, =6,, 
@,,X,+ : + 21, xn =b, 
Bigs es Fay Xn t Xn ar = Opays 
Brgy Kyte Asn Xn + nes =Diyy 
G,,*, Fees Try xn + Xnscren = Op 


The additional variables appear in the linear form with coefficients which 
are equal to zero. 

If t=a, the general linear-programming problem is a priori given in 
canonical form. In this case, n=n+(r—l); m=r. (We remind the reader 
that t is the number of nonnegative variables in the problem (2.1)-(2.4).) 

Now, let the mixed restraints be reduced to a system of linear equali- 
ties, but let t<a, i.e., the nonnegativity requirement does not apply to all 
the variables. In this case, it is simplest to pass to the canonical form by 
replacing the variables x,, which are not restrained by the nonnegativity re- 
quirement, by the difference x ,;=Xj—X}, where x)= 0, x; 0. 

However, this method will increase the number of variables. The ca- 
nonical form of a problem may be established differently, by a method 
which, generally speaking, reduces the number of variables and restraints. 

We now apply the linear equations restraining the choice of variables and 
express the x, which are not restrained by the nonnegativity requirement 
in terms of the remaining variables. We then substitute the resulting ex- 
pressions into the linear form and into the restraints which have not been 
used in the reduction procedure. 

The procedure leads to one of the following cases, depending on the 
linear form, the linear restraints, and the relationship between the number 
of restraints and the number of nonnegative variables: 

(1) The linear-programming problem is unsolvable. 

(2) The total number of variables and restraints is reduced. The linear- 
programming problem takes on the canonical form which, however, does 
not guarantee solvability. 
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9-2. We now give an example of reducing a linear-programming problem 
to canonical form. 
Maximize the linear form 
L=x,+x,+¢x,+.-, 
subject to the conditions 
ax.+ xX,+ *,— *%,+4%,= 4, 
x, +ax,+ *,—2x,—3x, = 3, 
X,+ *,—2x,—3x,+4 2x, <2, 
X,+ *%,+ax,—4x,—4x,< 1, 
x,20, x, 20. 
To reduce the mixed restraints to equalities, we shall introduce two ad- 
ditional variables 
xo 0, x, S>0. 


The restraints take on the form 
ax,+ X,+ X,— 4,4 4%, = 4, 
X,+ax,+ x,—2x,—3x, = 3, 
X,+ *,—2x,—3x, + 2x,+%, =2, 
X,+ xX, + ax,—4x,—4x,+%,=1, 
x20, x20, x,>0, x,>0. 

The variables not restrained by the nonnegativity conditions should now 
be expressed in terms of the remaining variables. Two cases are possible: 
(1) three of the four equations are solvable in x,, x,, x,; (2) there are no 
such equations. 

The determinant whose elements are the coefficients of x,,x,, and x, in 
the first, second, and fourth equations is equal to A=(a—1)*(@+2). For 
a~1land a3—2, these equations can be used to express x,,x,, and x, in 
terms of the nonnegative variables. For example, let a=0. We calculate 
x,, x, and x,, and substitute the resulting expressions into the linear form 
and the third restraint, which has not been used in the procedure. 

We obtain ; ‘4 

x, = 7% a%s— aX 
x= 14+ 5x,— %— 5% 
x, =3—% x, 2 x4 %y 


L=(143¢)+-4 x, (10—c)+4 x, (8—5e) — 4 x,(2—c), 


2x, + 11x, + x,—2x, =7, 
x20, x,>0, x, >0, x, >0. 
This is the canonical form of the problem*, 

Consider, now, another case, where no three equations of four can be 
chosen so that the resulting solutions can be used to express those variables 
which are not bound by the condition of nonnegativity. This case is en- 
countered when, e.g., a=—2. The problem in this case is stated as follows. 

Maximize the linear form 

L=x%,+%,+¢x%,+%, 
subject to the conditions 
—22x,+ %,+ X,— *,+ 4%, a= 4, 


¥,—2x,+ *,—2x,—3x, =3, 
* The constant (1+ 3c) in the linear form can be omitted since it does not affect the extremum of L, 
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¥,+ %,—2x,—3x,+ 2x,+%,=2, 
xX, *,—2x,—4x,— 4x, +%,=1, 
x0, x, 0, x, 0, x, 20. 

Here x,, x, and x, cannot be expressed in terms of the nonnegative 
variables, but the first and the second equations can be used to express x, 
and x, in terms of the remaining variables. After doing so and substituting 
the result into the remaining restraints and into the linear form, we obtain 


L= —7+(2+¢)x,—2x,+%,, 
— 6x, +3x,+%,=9, (5.1) 
—7x,—3x,+%,=8, 
X00, x,=70, 4,20, x, 20. 


The variable x, does not appear inthe restraints and thus remains un- 
specified. If, therefore, the coefficient of x, inthe linear form is not zero, 
the linear-programming problem does not have a bounded solution. If, 
however, c=—2, the system (5.1) is the canonical form of a solvable problem. 

The above order of computations is independent of the particular example, 
and may be employed whenever a linear-programming problem is to be re- 
duced to canonical form. 


§ 6. Geometrical interpretation of the simplest 
linear-programming problems 


6-1, Linear-programming problems with the number of variables ex- 
ceeding by two or by three the number of restraints can be interpreted geo- 
metrically in the plane or in three-dimensional space. (When we say that 
the number of variables is greater by two or by three than the number of 
restraints, we are considering the canonical form of the problem.) Intro- 
ducing elements of geometry of nz-dimensional space, we extend the geo- 
metrical interpretation given in this section to problems with any number 
of variables and restraints. This is called the first geometrical 
interpretation, and can be applied to linear-programming problems 
written in any form. 

The geometrical interpretation of a problem enables us to discuss in 
geometrical terms the various methods of solution, 

Consider the following particular linear-programming problem. 

Maximize the linear form 


L=¢,x%,+¢,%, (6.1) 


subject to the conditions 


@,,X,+2,,%, <5,, (6 2) 
Bu, X, + Bg at, dy: 
x, = 0, x, > 0. (6.3) 


The inequalities (6.3) specify the positive quadrant of the x,Ox,-plane. 
The. equation 


@;,X,+4;,%, = 5, (6.4) 
associated with the i-th restraint defines a straight line on the x,Ox,-plane, 
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and the inequality 
G;,%, + 2;,%, =D, (6.5) 


specifies the half-plane whose direction relative to the line (6.4) is given by 
the vector (—a,;,, —a,;,), drawn from any point on this line. 

The domain of definition of the linear form in the problem (6.1)-(6.3) is, 
thus, the intersection of the half-planes (6.5) (i=1, 2, ..., m)and the positive 
quadrant of the x,Ox,-plane. 


The equality 
L=c,x,+¢,%,, 


for L=const, is the equation of a family of parallel lines. The parameter 
L of the family is proportional to the distance from the origin to the cor- 
responding line. 

In Figure 1.1 we show a case where the domain of definition of the linear 
form is a polygon. . The lines bounding the polygon OABCD inthe x,Ox,- 
plane correspond to conditions (6.2) and (6.3), equalities being assumed 
there. Striation indicates the direction of the half-planes defined by the in- 
equalities (6.2) and (6.3). The direction of the line MN is defined by the 
vector (c,,¢,) (the vector (c,, c,) is perpendicular to the line MN), The vector 
(c,, ¢,) also points in the direction of increase of the linear form. The linear- 
programming problem, i.e., the determination of the coordinates of the 
point at which the extremum of the linear form (6.1), subject to (6.2) and 
(6.3), is attained, can be interpreted geometrically for n=2as follows. 


z, W 


FIGURE 1,1 FIGURE 1,2 


We draw the line L=c,x,+¢,x, through the restraint polygon (the domain 
of definition of the linear form) and translate it parallel to itself in the 
direction of increasing L, if the linear form is to be maximized, or in the 
direction of decreasing L, if the linear form is to be minimized. Two 
cases are possible. In the case shown in Figure 1.1, parallel translation 
will bring the line into the position M’N’, where it has only one common 
point with the polygon, the common point being the vertex A. This point 
specifies the unique solution of the linear-programming problem (in Fig- 
ure 1,1 this is the maximum of the linear form). It may happen, however, 
that the line MN is parallel to one or to two sides of the polygon. This is 
shown in Figure 1.2. In this case the extremum is attained at all the points 
of the corresponding side of the polygon. In Figure 1.2, at all points of 
the side AB of the polygon ABCDE parallel to the line MN the linear form 
is maximized, and at all points of the side ED|\| MN, it is minimized. The 
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linear-programming problem may, thus, have either one or an infinite num - 
ber of solutions. If two vertices extremize the linear form, all the points 
of the segment joining these vertices are solutions of the linear-program - 
ming problem. 

Figure 1.3 corresponds to the case of an unsolvable linear-programming 
problem: the restraints appear to be contradictory. 
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FIGURE 1,3 FIGURE 1,4 


Figure 1,4 shows an unbounded domain of definition of the linear form. 

If AB|| MN, the linear form attains a finite extremum at all points of the ray 
AB. If the domain of definition of the linear form is modified by rotating the 
ray AB about the point A, we have two cases: either the linear form is not 
bounded at the feasible values of the variables, or it is maximized ata 
single point. The first case is depicted by the ray A8’—the dashed line 
(Figure 1.4), The second case is depicted by AB’ — the dash-dot line (Fig- 
ure 1.4). 

6-2. Until now we have considered the geometrical interpretation of a 
linear-programming problem in two variables and m inequality restraints 
of the form (6.2). If this problem is reduced to canonical form, the num- 
ber of variables of the new problem will be n= m+2, and the number of 
restraints (equalities) will be m. We thus have n—m=2. 

It can easily be seen that the geometrical interpretation of linear-pro- 
gramming problems given in 6-1 applies only when za—m=2, n and m being 
arbitrary. We express all the variables in terms of only two of them, say 
x,and x,, and rewrite the linear form and the restraints as follows: 

L= Ox, +6%,, 
Ly = Oi, + AX, —As, s=3,4, ..., 4, 
%,220, j=l, 2, ..., 2. 
The lines x,=0 (aX, + @y5X, = a) and the axes define a convex polygon (a con- 
vex polygonal set) in the x,Ox,-plane. The conditions x,=0 specify the 
direction of striation of the polygon sides. We have thus specified the range 
of all the possible pairs (x,,x,) amongst which the points maximizing (or 
minimizing) the linear form must be chosen. The coefficients of the linear 
form define a family of parallel lines and the direction in which L increases. 
We choose from this family a line intercepting the polygon and translate it 
in the direction of increasing (decreasing) linear form, until it has no com- 
mon points with the polygon. The limiting position of the line specifies the 
maximum (minimum) of the linear form. 

As illustrative is the geometrical interpretation of the linear-program - 

ming problem for three variables, whenthe restraints are inequalities, or 
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when the number of variables exceeds by three the number of restraints 
(n—m=3) and the problem is given in canonical form. The restraints de- 
fine in space a convex polyhedron (a convex polyhedral set), The linear- 
form coefficients specify a family of parallel planes H and the direction in 
which L increases. To solve a linear-programming problem, the plane H 
intercepting the polyhedron should be translated in the direction of increas- 
ing linear form, if maximization is required, or in the direction of de- 
creasing L, if minimization is required, until it has no common points with 
the polyhedron. The limiting position of the plane specifies the solution 

of the problem. 

Reasoning geometrically shows, as before, that the form is extremized 
at the vertices of the polyhedron. If the form is extremized at more than 
one point, it is extremized over the entire edge or face of the polyhedron 
parallel to the plane defined by the linear-form coefficients. 

In the following we shall see that all the features of linear-programming 
problems and their methods of solution can be explained within the frame- 
work of this geometrical interpretation. 

6-3. We now give another geometrical interpretation of linear-program - 
ming problems. The second geometrical interpretation illus- 
trates methods of solution of canonical problems. We shall consider prob- 
lems with two restraints and any number of variables. These problems 
can be visualized in three-dimensional space. Suppose it is required to 
maximize the linear form 


L=c,x,+¢,x,+...+¢,%, (6.6) 
subject to the conditions 
a,x, +4,,%,+...+4,,%,=5,, } (6.7) 
@,.%,+4,,x%,+...+4,,%, =9),, 
x, 20, j=), 2, 000, A 


We introduce the new variables 


“= a,,%, +... +4,,%,) 
U,=4,,%,+.-.- +425) (6.8) 


U,=C,X, 4... CyX, 


Relationships (6.8) define a transformation of the a-dimensional space of 
the variables x,, x,, ..., x, into the three-dimensional space of the variables 
u,, u,, U5. The positive semiaxes Ox,, Ox,, ..., Ox, of the z-dimensional 
space are transformed into rays issuing from the origin. Indeed, the co- 
ordinates of the positive semiaxis, Ox,, satisfy the conditions x,=x,=...= 
=x,=0. The image of the Ox,-axis in the u,, u,, 4,-space is therefore de- 
fined by the relationships 


U = 4,,%,, U,=4,,%,, U,=c,x,, Omx, < oo. (6.9) 


Hence, the image of the Ox,-semiaxis is a ray in the three-dimensional 
space Of g,, u,, u,. Similarly, the positive semiaxis, Ox,, 
MS Ny Kg SH... = HX, =0, OX, < 00,7 
is transformed into the ray 
Ub, = 4, Xp, 
Uy = A,,X%_, Uy = C,%,, 
0< x, < 00. 
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Figure 1.5 shows, in the coordinates u,, u,, 4,, the rays A, correspond- 
ing to the positive semiaxes in the x,, ...,x,-space. All the rays A, define 
a convex polyhedral cone K, the image of the orthant* x,>0 (j=1, 2, ..., a) 
in the x-space. The orthant x,20 is a convex set. It can easily be shown 
that a linear transformation transforms a convex set into a convex set, 
Hence the convexity of the polyhedral cone. 


FIGURE 1.5 


Let A, be a three-dimensional (and, generally, (m+1)-dimensional) vec- 
tor with components a,,, a,,, c, (in general, with components a,,, ay), ..., @ny, ©), 


Aj= (G17, Gqy, Cy)" (A, = (@,,, Oyj) +20) Gps, C;)"). 


The first two (m) components of the vector Aj coincide with the compo- 
nents of the restraint vector, and the third ((m+1)-th) component is equal to 
the corresponding linear-form coefficient. We shall refer to vector A, as 
the augmented restraint vector. Augmented restraint vectors 
define the direction of the A, rays, the images of the positive semiaxes in 
the x,, x,, ..., x,-space. It follows from (6.8) that the positive orthant in 
the x-space is transformed in the three-dimensional space into a convex 
cone defined by the augmented restraint vectors 

re ere 

6-4, To study the features of linear-programming problems, we should 
consider not the image of the positive orthant, but rather the image of the 
domain of definition of the particular problem in question. It is, therefore, 
expedient to consider, together with the rays A, (the images of the positive 
semiaxes), also the line Q defined by 


“un=Ob,, 
u, = 6,, 
uy =g (—0% <q< oo), 
where 6, are the components of the constraint vector B. 
The line Q is parallel to the Ou,-axis, and its projection onto the (u,, u,)- 
plane is the constraint vector 8. In Figure 1.5, Q is vertical. Its projec- 
tion onto the u,Ou,-plane has the components (8,, 4,). 


* A positive orthant is generally defined as the n-dimensional analog of the positive octant, i.e., the 
set of vectors X with nonnegative coordinates, 
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We shall show that the image of the domain of definition of a linear- 
programming problem in the u,, uw,, u,-space is the intersection Q, of the line 
Q and the convex polyhedral cone K. Indeed, for any point a=(u,, u,, &,) E Qy, 
we have 

° B= 4X, +...$4,,%,=),, 


Ue=a,x,+...+4,,7%,=5,, 


Since Q, is a subset of Q. Moreover, for all the points X€Q,y which are 
the images of the points w€Q, 


x, 0, x, 50, ..., x, 20, 


since the corresponding points uw belong to the polyhedral cone, which is 
the image of the positive orthant in the x-space. To each point of the inter- 
section of the line Q and the cone XK there correspond certain points from 
the domain of definition of the linear form. The converse is obviously also 
true: each point of the domain of definition of the problem is mapped onto 
a point of the set Q,, which is the intersection of the polyhedral cone and 
the line Q. The set Q, is a segment, a ray or a line. 

To each point X€Q, there corresponds a certain u,=g. But 


@,=C€,x,+...+¢,%, 


specifies the value of the linear form L. Hence the parameter gq of the 
line Q corresponds to the magnitude of the linear form. If the line Q con- 
tains inside points of the convex polyhedral] cone, the boundary of the cone 
has only two points of intersection with this line. One of these corresponds 
to the maximum of the linear form, the other to the minimum. If the line 
Q is tangent to the cone, i.e., if it belongs to one of its faces, the upper 
point of contact specifies the maximum of the linear form, and the lower 
its minimum, 

The linear-programming problem, i.e., the problem of extremizing 
the linear form (6.6) subject to (6.7), is reduced to the determination of the 
extremum (upper in the case of maximization and lower in the case of mi- 
nimization) point of intersection of the line Q with the convex polyhedral 
cone, 

If Q does not intersect the convex polyhedral cone defined by the aug- 
mented restraint vectors A,, A,, ..., A,, the domain of definition of the 
linear-programming problem is empty. In this case the problem is ob- 
viously unsolvable. If the intersection of Q and the convex nolyhedral cone 
is not bounded (from above for a maximization problem, and from below 
for a minimization problem), the problem in question is unsolvable because 
the linear form is not bounded in the domain of its definition. 

In Chapter 4 the two geometrical interpretations are generalized for 
linear-programming problems with any number of variables and restraints. 


§ 7. Economic interpretation of linear-programming problems 


7-1, Economic problems constitute the main source of linear-program - 
ming problems. In fact, linear programming arose as a tool for filling the 
requirements of economics. It is, therefore, advisable to supplement the 
geometrical interpretation of the linear-programming problem by the cor- 
responding economic interpretation. 
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Consider the scheduling of operations of an enterprise or of a group of 
enterprises producing some homogeneous commodity. The production of 
the commodity requires several types of raw materials, certain machine- 
tools, certain man-hour potentials of laborers of various qualifications, 
power, fuel, transport, etc. We shall combine all these aspects under the 
heading of production factors. Let the number of factors specifying the 
production technology be m. Each of these is, as a rule, available in the 
enterprise in limited quantities. 

Each feasible set of production factors ensures a finite production output 
in unit time. We shall use the term ''mode of production" to designate asys- 
tem of m+1 numbers (an (m+ 1)-dimensional vector) specifying the require- 
ments of each production factor in unit time and the corresponding output. 

In each problem several modes of production can be indicated, each 
utilizing the production factors in manufacturing the commodity differently. 
The following scheduling procedure of an enterprise is feasible. 

First prepare (or assume as known) several modes of production, i.e., 
several sets specifying the requirements of each production factor in unit 
time and the corresponding output of the commodity. The consumption of 
production factors of different categories and the output of the commodity 
depends on the time during which the enterprise operates according to each 
of the feasible modes of production. 

The scheduling problem is, thus, to determine the time to be assigned 
to each individual mode of production. 

The optimal schedule is that program in which the consumption of each 
production factor does not exceed its admissible value, and the output of the 
enterprise attains the highest level. Obviously, the optimal plan is the one 
where the greatest number of production modes are subjected to analysis, 
and processing. 

During the formulation of the scheduling problem (in the previous sense) 
the following assumptions, which appear quite natural in the general con- 
text, should be made. 

If one mode of production ensures an output of ¢, units of a particular 
commodity in unit time and another mode, ec, units, then by scheduling the 
enterprise to operate for x, time units according to the first mode and x, 
time units according to the second mode we ensure an output of ¢,x,+c,x, 
units of the commodity. There is, obviously, no need to assume that the 
time intervals x, and. x, are consecutive. The enterprise may be simul- 
taneously scheduled to operate according to different modes of production, 
with some modes being replaced by other modes at definite intervals. The 
time intervals x, and x, should thus be interpreted as the specific weights 
of the particular production modes in the production program of the enter- 
prise, 

The time of switching over from one mode of production to another, i.e., 
the idling time, is not taken into consideration here. 

7-2. We now translate the scheduling problem into the language of ma- 
thematics. 

Let c, be the number of units of the commodity produced by the plant in 
unit time of operation according to the j-th production mode, where the 
consumption of the first production factor is a,,, of the second production 
factor a,, ..., of the i-th — a,,, ..., and of the m-th production factor a,,. 
Let, moreover, the quantities 56,, 6,, ..., 6, denote the available resources 
of the various production factors. The quantities 6, limit the permissible 
consumption. 
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Assume that in our scheduling we have at our disposal n modes of pro- 
duction. Let x, be the time interval during which the plant is scheduled to 
operate according to the /-th mode. The total output of the plant is then 
determined by 


L=¢,x%,+¢,x,+...+¢,%,. (7.1) 


The restraints imposed on the consumption of the individual production fac- 
tors are written in the form 


G,,%,+¢4,,x,+...+2,,%, 0), 
eT ee ee bi si aera ae oy (7.2) 
BuyX + SgaXy tees $2,,%,55,. 
The time intervals are, obviously, nonnegative, i.e. , 
x,20, j=l, 2, ..., 2. (7.3) 


Observe that the scheduling problem has been reduced to a linear-pro- 
gramming problem. The choice of the program is the determination of the 
set of numbers ~¥x,, x,,...,x,, for which conditions (7.2) and (7.3) are satis- 
fied and the form (7.1) is maximized. 

The terminology of linear-programming problems used in economics is 
also convenient when treating other problems described by the same formal 
apparatus. The economic interpretation of the linear-programming prob- 
lem will be useful later on for an intuitive substantiation of the various con- 
cepts introduced in algorithms. The vectors 


A, =(4,,, vere a B=(6,, eee, b,,)7 


were previously termed the restraint vector and the constraint vector, 
respectively. In economic interpretation, the components of the restraint 
vector specify the demand in individual production factors in unit time for 
the corresponding mode of production, andthe components of the constraint 
vectors give the supply of the individual factors limiting the actual demand. 

In terms of the production problem, the vectors A, (j=1, 2,..., a) and 
the vector B should, naturally, be interpreted as the demand vector 
and the supply vector, respectively. 

The j-th mode of production is defined by the vector A,, whose first m 
components coincide with the components of the restraint vector A,, and 
whose last component is ¢,, i.e., the output in unit time. According to the 
definition introduced in §6, A, will be considered as an augmentation of the 
vector A, Augmented vectors characterize entirely the modes of produc - 
tion, both specifying the inputs and the outputs. 

A set of numbers x,, x,, ..., *¥, satisfying the restraints of a linear-pro- 
gramming problem was called (see §2)a program. This term, which 
isself-explanatory inproblemsof production scheduling, is retained for all 
linear-programming problems regardless of their context, The same ap- 
plies to the term optimal program and to certain additional terms 
which will be introduced in the following. 

7-3, The problem stated here in terms of production scheduling may be 
modified to fit the requirements of the static model of national -economy 
planning. 

Consider a model of interaction between various branches of industry in 
an economic region, Let the economy of the region be shaped by a industrial 
branches. For simplicity we assume that the region is economically 
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selfcontained. The production schedule for the output commodity* of each 
branch is assumed to be known. Part of the output is exported to other 
regions and part is locally consumed, All the branches of industry are in- 
terdependent ard normal functioning of each branch is impossible unless 

an orderly supply of production means from other branches is maintained. 
The production program for individual branches of the economy and the 
interaction between the various branches should ensure minimum labor 
demand (in man-hours and in machine facilities) for the production of output 
commodities in quotas specified and according to a preset schedule. 

We now translate this problem of planning the interaction between the 
branches of industry in a selfcontained economic region into formal lan- 
guage. In order to be specific, we shall assume that an annual plan is 
drawn. We introduce the following notations. 

Let x, be the total output of the i-th branch during the period specified; 
x, the total amount of materials supplied by the i-th branch to the j-th 
branch in one year; and y, the annual quota of output commodity of the i-th 
branch, 

The parameters x, and y, like x, are measured in natural units; y, is 
specified by government economic programs which, in particular, take into 
consideration the requirements of the region. 

The problem of planning regional economy under a static model consists 
of drawing quotas for the various branches of industry which would fill the 
demand in products of any branch in all other dependent branches, and would 
also satisfy the state quotas for the output commodity of each branch. 

Applying mathematical notations, we represent the requirements of the 
problem as a system of inequalities 


£2 Bey tI (7,4) 


The ratio aay specifies the amount of the commodity, produced by 


the i-th branch, required for producing a unit of commodity released by the 
j-the branch. In these new notations, conditions (7.4) are written in the 
form 


x5 D> 2, XP Yj (7.5) 


or, in matrix form, 


where / is the unit matrix of order na; A=|a;,||, is the interaction matrix; 
and X and Y are n-dimensional vectors with components x, and y,, res- 
pectively. The vector X is the production vector, and Y is the output com- 
modity vector. 

The matrix J—A is generally referred to ineconomics asthe Leontieff 
matrix. Given the vector X, W. Leontieff calculatedthe vector Y for 
some time in advance, proceeding from the values of a,; established on 
the basis of previous experience. 

Until now we have assumed, implicitly, that there exists only one tech- 
nological process for utilizing the products of various regional branches of 
the economy in manufacturing the commodity of any given branch. It seems, 
however, that alternative interaction models also exist /65/. 


* We remind the reader that an output commodity is a commodity which is not used as raw material in some 
other production processes (consumer items, goods for export, military equipment, etc.). 
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Normal output of each branch can generally be ensured for various re- 
lationships with the other branches of the economic region. A unit output 
of each branch may be produced for different patterns of supply from the 
other branches. The labor requirements for producing one unit of a com- 
modity obviously depend on the pattern chosen. 

Let each branch function under ? different production modes. Each 
technological process for producing the commodity of the /-th branch is 
characterized by the set of numbers a!) (i=1, 2, ..., a), and c/ The matrix 
A(s) = |] aff ||, defines the industrial interdependence for the s-th mode of pro- 
duction. The vector c=(ci, ...,c) characterizes the labor requirements 
for unit output of each branch of the economy scheduled for the s-th produc- 
tion mode, 

As the technologically feasible modes of prcduction in each branch be- 
come more numerous, i,e., the range of values assumed by the index s 
increases, the scope of feasible programs becomes wider and the labor re- 
quirements corresponding to the optimal plan drop closer to the minimum. 

The best program is determined by the vectors xX, minimizing the linear 
form 


f 
L=> CX, (7.7) 
s=1 
subject to the conditions 
t 
2 (1A) X,2>Y, (7.8) 
X, 20. (7.9) 


Here we have assumed that all the branches of the economy have the 
same number of technologically feasible production modes. 

If, however, the j-th branch has /!, modes of production, the problem is 
stated as follows. 


Determine the variables x,, minimizing the linear form 


nil 
a2 2 cf Xs (7.10) 
subject to the conditions 
& nid 
Bai DB ols my (7.11) 
x, 0, (7,12) 


i, j=l, 2,....4 s=1,2, cee dy 


Here aj/x,, is the output of the i-th branch required for producing «,, 
units of the commodity of the j-th branch, according to the s-th techno- 


s) 
the quota of the j-th branch; )) x,; is the total output of the i-th branch; 


‘ 
logical method; 3 aj)x,, is the output of the i-th branch required for filling 
$=1 &g 


cf x.) are the labor requirements for manufacturing x,, units of the commo- 


nf 
dity of the j-th branch by the s-th technological method; and p2 > cf x, is 


is=1 
the total labor requirement for filling all the regional quotas by all the 
feasible technological methods. 
It can be shown (see Exercise 4) that given a single technological mode 


31 


of production (s=1), the optimal program reduces the inequality (7.8) to an 
equality, The program balancing the input and the output, i.e., the pro- 
gram X satisfying the equation (/—A)X=Y¥Y), is in this case the optimal 
program for any demand vector C=0/83/. In the general case, however, 
the optimal plan does not reduce inequalities (7.8) or (7.11) to equalities. 

Observe that the problem of national-economy planning, like the problem 
of industrial scheduling, is a general linear-programming problem. We 
have already seen that certain theoretical concepts and some principles of 
method design in linear programming have a manifest economic background. 
By formulating the general postulates of linear programming in terms of 
economic concepts we acquire the intuition for assessing new trends in the 
development of this discipline. 


EXERCISES TO CHAPTER 1 


1, Proceeding from the arguments of 3-1, find the measure function and the restraints for an allocation 
problem dealing with the supply of an inhomogeneous product to a destinations from m production origins, 

2, Formulate and reduce to a linear- programming problem the following problem of allocation for a 
sowing area. 

Sowing areas of collective farms in a certain region are $, Sg, ..., Sg, respectively. According to a 
master plan, the crops should be produced in the region in the proportion p,:p,:...:P_,- The expected 
output of an ¢-th crop from the field of the j-th farm is a;y units from 1 acre. Determine the fraction of 
sowing area to be allocated to each crop in each farm to ensure maximum output with the prescribed crop 
ratio, 

3. Solve graphically a scheduling problem (see 7-2) for the case when the supply vector B=(50, 20, 60, 

90)7, and the augmented demand vectors of two production modes are, respectively, 


A,=(2, 2, 5, 1, 1)7, 
A,=(5, 1,6, 10, 1)*. 


4. Prove that in a model of national-economy planning (see 7-3) limited by a single technological factor 
(s=1), the optimal plan reduces the inequality (7.8) to an equality. 
5. Find the canonical form of the problem maximizing the linear form 


L=%z,—x, 
subject to the conditions 
—2x, +ax, <2, 
Xx, ~— bx, & 3, 
x, +%, Se, 
dx, +3 <1, 
x, = 0. 


Analyze the different cases depending on the value of the constants a, 6, ¢, and d, 


6. Applying the first geometrical interpretation of the linear-programming problem, determine the con- 
stants in Example 3d for which 


(a) the solution is unique; 

(b) the restraints are inconsistent; 

(c) the domain of definition of the linear form is not bounded, but the problem is solvable; 
(d) the linear form is not bounded in its domain of definition; 

(e) the problem has an infinite number of solutions. 


7. Proceeding from the geometrical interpretation of the linear-programming problem, determine the 
range of the linear form 


L =z£,— Xe, 
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if the variables are limited by the conditions 
X,+3xg+ r+ 2% +2x,=6, 
—2x,+ Xe t+ 3x, — 4x, 4+ x,=2, 
5x,—2x, + X,+2%,—3x,=— 4, 
xj0, j=l, 2,3, 4, 5. 


8. Applying the second geometrical interpretation of the linear- programming problem, indicate the 
range of the parameters a, 6, and ¢ in which the following problem is solvable; 
Maximize the linear form 
L=x,+3x, +cx,—x, 
subject to the conditions 


ax,—2x,+ X,—3z,=5, 
Xg+2x%,+ %,=4, 
x20, jal, 2 3, 4 
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Chapter 2 
. CONVEX POLYHEDRAL SETS AND LINEAR PROGRAMMING 


The general linear-programming problem is concerned with determin- 
ing the extremum (minimum or maximum) of a linear function whose 
variables are restrained by several linear relationships (equalities or inequa- 
lities). The set af points defined in the space of the variables by these re- 
straints is a many-dimensional analog of the two-dimensional convex poly- 
gonal domain whose boundary is a polygonal line with a finite number of 
sides. These sets are generally called convex polyhedral sets, 

The linear-programming problem thus deals with the analysis of a linear 
function defined on some convex set. Hence the close relationship between 
linear programming and the theory of convex polyhedral sets. 

In this chapter we present some elements of this theory which are essen- 
tial for substantiating most of the results of linear programming (§1-§ 3). 
Although our approach to the theory of convex polyhedral sets differs con- 
siderably from other known methods used in discussing this theory (see /23, 
24/, and also /107/), we believe that it is more natural and illustrative. 

In § 4 the results of the previous sections are used to establish several 
important theorems of linear programming. The concluding §5 gives two 
different geometrical interpretations of linear-programming problems 
which, we hope, will be conducive to a better understanding of the subject. 


§ 1. Convex polyhedral sets 


1-1. Consider all-the feasible programs of an arbitrary linear-program - 
ming problem in a variables. In other words, considera set of points 
(vectors) X=(x,,...,x,) defined by an arbitrary system of linear restraints 
(equalities and inequalities): 


(D, Y= Ddyxj<d;, t=1,2,...,8; (1.1) 
j= 
(Dj, X= d; jx, = dj, i=s+l, s+ 2, ee ey s+, (1.2) 
f= 


We do not write the conditions of the form x,20 separately, as is usually 
done in formulating the restraints of a linear-programming problem. 
These conditions (if specified) are included in the general system of inequa- 
lities (1.1). Let M be the set of points X satisfying (1.1) and (1.2). 

We shall assume that the system (1.1), (1.2) is consitsent, i.e., the 
set M is not empty. In this section we propose to establish several simple, 
and yet important properties of the set M which consists of all the feasible 
programs of the general linear-programming problem. 


34 


Theorem 1,1. The set M is a convex closed set*. 
Proof. 1. Let X,,X, be any two points of M; let O0<aq@i. In this case 
the point X=aX,+(1—a)X,EM. Indeed, (D,, X)=a(D,, X,)+(i—a)(D,;, X,). Hence 


(D;, X)<ad;+(l1—a)d;=d;, i=1,2,...,8; 


(D,, X)=ad,;+(1—a)d,;=d,, f=s+1,s+2, ..., s+, 
i.e. , XEM. 
By definition the set M is, therefore, convex. 
2. Let X,,k=1,2,... be an arbitrary sequence of points in M converging 


to the point xX. Since the scalar product of vectors is a continuous func - 
tion, we have 
J (D,, X,) = (Dj, X) (1.3) 


for any i. By assumption, 


(D;,, X,<d;, i=1,2,...,8, 

(D;, X,)=d;  i=s+1, s+2,...,s+#, (k=1,2,...). 
In these relationships passing to the limit as k—+oo and applying (1.3), we 
observe that the point X satisfies (1.1) and (1.2), i.e. , belongs to the set 
M. The set M thus contains all its cluster points and as such is a closed 
set. This completes the proof. 

The structure of (1.1), (1.2) shows that the set M is actually the inter- 
section of the half-spaces (D,, X)<d;,i=1,,2,...,s with the hyperplanes 
(D,, X)=da;, t=s+1,s+2,...,8+4 in the a-dimensional space of the points 
(vectors) X=(x,, x,,...,*,) The definition of a hyperplane and a half-space 
for any finite-dimensional space is given in the Appendix, 1-7. 

The set comprising the intersection of a finite number of half-spaces 
and hyperplanes (provided this intersection is not empty) will be calleda 
convex polyhedral set. M is thus a convex polyhedral set. 

The preceding definition indicates that any convex polyhedral set can be 
defined by a consistent system of restraints of the form (1.1), (1.2). 

Since in the following we shall deal only with convex polyhedral sets, 
the adjective convex is dropped wherever no confusion can arise, The 
system (1.1), (1.2) generating the convex polyhedral set M is sometimes 
called the restraint system of the set, 

A convex polyhedron is defined as a bounded convex polyhedral 
set. If the set M is bounded, (1.1), (1.2) define a convex polyhedron in an 
n-dimensional space. 

According to the preceding definitions, all the feasible programs ofa 
linear-programming problem can be described as the polyhedral re- 
straint set (orthe restraint polyhedron) of the problem. 

1-2, We now establish the relation between the dimensionality of the 
polyhedral set M (see Appendix, 3-4) and the properties of the restraints 
(1.1), (1.2) generating this set. First we give some definitions. 

The i-th constraint of the set (1.1), (1.2) will be called a rigid con- 
straint of the polyhedral set M if any point of M satisfies the correspond- 
ing equality, i.e., for XEM,(D,, X)=4;. 

Any of the conditions (1.2) is obviously a rigid constraint imposed on mM. 
Some of the conditions (1.1) may also prove to be rigid constraints. 


* For the definition of a convex set and a closed set, see Appendix, § 3, 
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For example, let us consider a polyhedral set M’ defined by the inequa- 
lities xt x,<1, 
2x,+ X%, = 2, (1.4) 
—3x,—2x,<—3, 
x,<l. 


Let X=(x,, x, *x,)€M’. Adding the first two restraints in (1.4) and comparing 
the result with the right-hand side of the third inequality of the set, we 
Opiare — 3x, — 2x, =— 3. 

The third inequality of (1.4) thus imposes a rigid constraint on the polyhedral 
set M’. 

The i-th constraint of the system defining the polyhedral set M will be 
called a nonrigid constraint of this set if a point XE€M exists such 
that 

(D,, X)<d). 


The system of constraints defining a polyhedral set can thus be divided 
into two subsystems, one comprising rigid, and the other nonrigid con- 
straints of the given set. Nonrigid constraints are, obviously, provided only 
by the inequalities. In the preceding example, the fourth inequality im- 
poses a nonrigid constraint on M’, since the point (Xj, Xp, %) = (1, 0,0) EM’ 
and x,=0<1, It can be easily verified that the first two inequalities of (1.4) 
are actually rigid constraints of M”’, 

A system of linear constraints comprising conditions of the form (1.1) 
and (1.2) is said to be linearly independent if the corresponding sys- 
tem of vectors D, is linearly independent. The rank of a system of linear 
constraints is the rank of the matrix constructed from the vectors D, asso- 
ciated with the given system. 

We now proceed with the statement and the proof of the dimensionality 
theorem of polyhedral sets. 


Theorem 1.2. The dimensionality @ of a polyhedral set M is given by 
e=n—d, (1.5) 


where o is the rank of the system of rigid constraints of this set. 

Proof. Let the set of subscripts of the rigid and the nonrigid constraints 
imposed on M be denoted by £, and £,, respectively. By assumption, the 
convex polyhedral set M is the intersection of the hyperplanes 


(D;, X)=d, EB, (1.6) 


o of which are linearly independent. (A system of hyperplanes is said to be 
linearly independent, if the direction vectors of these hyperplanes 
are linearly independent.) Hence, according to the definition of dimension- 
ality of a convex set 


Qa n—o. (1.7) 


Consider a system of linear homogeneous equations 


(D, X)=0, i€Z,. (1.8) 
Since the rank of the system of vectors D, i€E,, is a, there exist a—o 
linearly independent vectors X,, i=1,2,...,2—0, satisfying equations (1.8). 
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Moreover, for any i€£, there exists a vector Y, satisfying (1.6) so that 
(D;, Y;)<d;. 
Taking 


where q is the number of nonrigid constraints imposed on M, we have 


(D;, Y,)<d; (1.9) 
for all i€£,. Since the set M is convex, Y,E€M. It follows from (1.9) that 
for any i=1,2,...,a—0 and sufficiently small e 

Y,t+ex,EM. 


Now, assume that the polyhedral set M is the intersection of some 
hyperplanes 
(A, X)=A,, G=1,2,...,4 
Since 
Y,Ee Mand Y,+eX,E M, 


(Ay, X) =F (Me ¥y+2X)— (Ae Y= Z eA) =0. 
The system of homogeneous equations 
(A, X)=0, a=1,2,...,f 


thus has a—o linearly independent solutions X;. Hence, among the vectors 
A,, a=1,2,...,/there are not more than oa linearly independent vectors. 
Thus, we have established that any system of hyperplanes whose intersec- 
tion contains the convex polyhedral set M comprises not more than o li- 
nearly independent hyperplanes. Therefore, 


Qpn—o. 


Comparing this result with the inequality (1.6), we obtain (1.5). This com- 
pletes the proof. 

Consider now the polyhedral set M’ defined by (1.4). The first three in- 
equalities in (1.4) impose rigid constraints on M’. The first two constraints 
are linearly independent, while the third is a linear combination of them. 
Hence, in this particular case, o=2. The dimensionality of the polyhedral 
set M’ is therefore 

Q=n—o=38—2=1, 


The system of constraints defining a polyhedral set may be reduced to 
an equivalent form with each inequality imposing a nonrigid constraint and 
all the equalities being linearly independent. To this end it suffices to re- 
place, in each rigid inequality constraint, the inequality by an equality. 
Then, those equations of the system which are a linear combination of 
others, should be eliminated. The system thus obtained is equivalent to the 
original system inthe sense that both define the same polyhedral set. As- 
suming that these manipulations have been carried out, and that (1.1), (1.2) 
is the resulting system, then, according to Theorem 1.2, the dimensionali- 
ty 9 of the set Mis 


g=n—t. (1.10) 


1-3, In what follows we shall use the term "face" of an arbitrary 
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polyhedral set. A subset G of a polyhedral set M is calleda g-dimen- 
sional face of M if 

(a) Gis g-dimensional; 

(b) from X=aX’+(1—a) X"EG, 0<a<land X’, X*EM it follows that X’, X"EG. 

It is easily observed that for qg=0 this definition reduces to the defini- 
tion of an extremum point of the set M. Extremum points of a polyhedral 
set are, naturally, referred to as vertices ofthis set. Thus, the zero- 
dimensional face of M and the vertex of M are equivalent concepts. 

The edge ofa polyhedral set is defined as any one-dimensional face 
of the set. 

The face of a polyhedral set M can also be defined in terms of (1.1), 
(1.2), iie., a g-dimensional face of the set M is any g-dimensional 
polyhedral set generated by restraints obtained from (1.1), (1.2) when 
some of the inequalities are replaced by equalities. Observe that by Theo- 
rem 1,2 the number of linearly independent rigid constraints of a g-dimen- 
sional face is ra—g. 

Theorem 1.3. The two definitions given above of a q-dimensional face 
of a polyhedral set are equivalent. 

Proof. 1. Let G bea g-dimensional face of a polyhedral set M in the 
sense of the first definition. Let &, be the set of subscripts ij so that 


(Dj, X)=d; 

for all XE€G. Obviously E,>F,, where &, is the set of subscripts of the 
rigid constraints of M. Let G’ be a polyhedral set whose restraints are ob- 
tained from (1.1), (1.2) when the inequalities for i€£, are replaced by equa- 
lities (this obviously refers only to those i€&, associated with restraints 
of the form (1.1)). We shall show that G=G’. Let X bea point in GO’; X, 
is a point in G such that 

(D,, X,)<4;, IGE, (1.11) 

(D;, X,)= 4; i€ Eg. (1.12) 


The existence of this point follows from the definition of the set Bg. 

Let X’=X,+e(X,—X), e>0. Obviously, X’ satisfies the equalities (1.12) 
for any e and the inequalities (1.11) for sufficiently small e. Hence, for 
sufficiently small e,X’€Gt Thus, X,X’€GCM,X,EG, where 

l ’ 5 
xX,= ee xX + Tre Xx. 
In this case it follows from the definition of a face of a polyhedral set 
(condition (b)) that ’ 
xX’, XEG. 

An arbitrary point X€G’ thus belongs to G, i.e., G’cG. The converse, 
GcG’, follows immediately from the definition of G’. Thus @G coincides 
with G’ and, correspondingly, satisfies the requirements of the second de- 
finition of a q-dimensional face of a polyhedral set. 

2. Now let G bea gq-dimensional face of M in the sense of the second 
definition. 

Let the system of restraints defining the polyhedral set G have the form 


(D,, X)<id;, IEG, 
(D,, X)=d, t€ Ey. 
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Consider a point X€@Q, which can be written in the form 


X=zaX” +(1—a) xX", O<a<cl, X’,XEM. 
Obviously, for any i 


(D,, X°) «&d;, (1.13) 
(D,, X") <& d,. (1.14) 


Let (€£,. Then 


ans (D,, X)=a(D,, X’) + (1—a)(D,, X") = di), 


! 
(D;, X)= = ([d,—(l1—ay(D,, X*)). 
Further, applying the i-th inequality of (1.14), we obtain 
1 
(Di X") > = (4, — (1—a) d,) =, 


Comparing this relationship with the i-th inequality of (1.13), we ob- 
serve that 
(D,, X°)=d;. 


Analogously, we obtain 
(D,, X") asd; 


Thus, xX’, X"E€G. Hence @ is a ¢dimensional subset of the set M satis- 
fying (b), i.e. , @ is a g-dimensional face of M in the sense of the first de- 
finition. This completes the proof. 

1-4, As a direct corollary of Theorem 1.3 we have the following impor- 
tant characteristic property of the vertices of an arbitrary polyhedral set. 


Theorem 1.4, A point XEM is a vertex of a polyhedral set if and only 
if among conditions (1.1), (1.2) there are a linearly indepenc -nt constraints 
which reduce to equalities at this point. 


To prove this theorem it is sufficient to recall that a vertex of a poly- 
hedral set is actually equivalent to a zero-dimensional face, and then to 
apply the second definition of a face of a polyhedral set. 

From Theorem 1,4 it follows, in particular, that any polyhedral set has 
at most a finite number of vertices. 

Indeed, a characteristic set of n linearly independent restraints from 
(1.1), (1.2) corresponds to each vertex of a polyhedral set M; different sets 
correspond to different vertices. Hence, the number of vertices M is at 
most Cle ‘ 

We shall now discuss the properties of edges (the one-dimensional faces) 
of a polyhedral set. 

Let I be some edge of a polyhedral set M. Write out all the rigid con- 
straints imposed on f. By assumption, there are exactly n—1 linearly in- 
dependent restraints among them. Hence, the edge [L is contained in the 
line of intersection of n—1 linearly independent hyperplanes of the form 


(Di X)= 4;, i€Er. (1.15) 


Let X,€M be on the line (1.15), and let the nonzero vector e point along this 
line (its components satisfy the homogeneous system corresponding to 
(1.15)). In this case the equation of the line in question can be written in 
the form 


X=X,+he, —o<ia< oo, (1.15' 
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Observe that this representation actually describes the general solution 
of the system (1.15), 

Since all the linearly independent rigid constraints of the edge I‘ can be 
obtained from the system (1.15), X, can be considered to reduce al! the re- 
maining restraints in (1.1), (1.2) to strict inequalities, i.e., 


(D;, X,)< 4), itEr (1.16) 


(those rigid constraints which are linear combinations of equations (1.15) 
can be neglected), 

Since the edge [lis the intersection of the line (1.15') with the polyhedral 
set M, it is a set of points X of the form (1.15'), where Ais determined from 


(D;,X,.)+4(Dj,,e)<d;, iGEp. (1.17) 


Analyzing the inequalities (1.17), four cases arise: 

1, (D,e)=0 for all i€E,. 

2. (D;,,e)=0 for all i€2p, and (Do, e}>0 for some i,. 

3. (D;,e)<0 for all i@Ep, and (D;,,e)<O0for some i,, 

4, (D;, e) can be either positive or negative. 

We shall discuss each of these cases separately. 

1. In this case the inequalities (1.17) impose no constraints on the para- 
meter 4. Hence, A may take on any value between-oo and oo, i.e., the edge 
lr coincides with the line (1.15'), Observe that in case 1 all the vectors D, 
for @£r are linear combinations of the vectors D, for!€Ep(see Appendix, 
Theorem 2,6), since, by assumption, the vectors D,, itfr are orthogonal to 
the nonzero vector e which is a solution of rank a—1lof the system 


(D;, e)=0, LEEP, 


In this case the rank of the system of restraints of the polyhedral set 
thus equalsa—1. In the following we shall see that this condition is not only 
necessary but also sufficient for the existence of a line edge. 

2. If A<0, then 

(D;, X,)+4(D;, e) <= (Dj, X,) < dj, 


i,.e., the parameter iis not bounded from below. 
The upper bound of dis, obviously, given by 


n= min 4 (Pn Xy) 


°  winey>o = (Dn ®) > %, (1.18) 


where the minimum is taken over those i¢£r for which (D,, e)>0 (by as- 
sumption, the set of these subscripts i is nonempty). 
The conditions constraining the range of A in this case are of the form 


—o<cAci, < oo, (1.19) 


i.e., the edge Lf is a half-line (ray) of the form (1.15') with 4 obeying the 
constraints expressed by (1.19). 

Consider the end point X=X,+4,e of the ray [. It follows from the defi- 
nition of A, that for some i’€E, 


(Dj: ? X) = d;, 
(Dy , e) > 0. 


It follows from the last inequality that the vector D; is not a linear combi- 
nation of D,for i€Ep. The point X¥ thus reduces to equalities a linearly 


and 


40 


independent restraints of the polyhedral set M(with i=i’ and /€Er), i.e., 
X is a vertex of M. The equation defining the edge [ can obviously be writ- 
ten here as 

X=X+ihe, 10, 


where X is a vertex of M. If we take e’=—e, the equation of the edge I 
can be rewritten in an equivalent form 


X=X+N(e", A SO. 
The nonzero vector e’ pointing along YF is generally called the direc - 
tion vector of the edge IP, 
Thus in case 2 the edge I’ is a ray issuing from some vertex of the poly- 
hedral set M. 


3. This case is analyzed precisely as the previous one. 
The edge Lf is a ray with the equation 


X=X+hre, A>0, 


where X=X,+4, eis a vertex of M, and e is the direction vector of the edge IP. 
Here 
d;—(D;,X,) 
4.= max Hoe d | 
wie<0 (Dye) 


4, In this case the parameter A for which X,+4e€ Mis bounded from 
above and from below is: 
MASA’. 
Here 
A’ sm min an >0, 
(D4. @)>0 by 
max (Pn Xo) 


<0. 
(D;, e) <0 (Dy e) 


The edge Lf is thus a segment whose end points are 
X=X, +e, K=X,+ie, 
and its equation is, consequently, given by 
X=AX+(1—A)X, OA, 


Reasoning as in case 2, we may easily show that X,X are vertices of the 
polyhedral set M. 

An edge of a polyhedral set which is a line or a half-line is called an 
unbounded edge. An edge that is a segment will be called abounded 
edge. 

Unbounded edges may, obviously, occur in unbounded polyhedral sets. 
All the edges of any polyhedron are segments withthe polyhedron vertices 
as the end points. Thus, using the various properties of edges of poly- 
hedral sets, we formulate the following proposition: 


Theorem 1.5. A subset Tf ofa polyhedral set M is an edge of M if and 
only if 

(a) f is contained in the line of intersection of n—1 linearly independent 
hyperplanes of the form (1.15); 

(b) Tisa line, avray, or a segment, 

(c) the end points of T, if any, are vertices of the polyhedral set M. 
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Proof, The necessary conditions of the theorem have already been 
established. It remains to prove that these conditions are sufficient. Let 
I satisfy the conditions of the theorem. Consider a polyhedral set M, whose 
restraints are obtained from (1.1), (1.2), where the inequalities for i€ Ep 
(leaci<js) are replaced by equalities. 

We shall show that [=M,. The polyhedral set M,is the intersection of 
M with the line (1.15). Obviously, [cM,. We shall verify that T=M,. If 
this were not so there would exist a point X’€M, outside [. Since xX’ belongs 
to the line (1.15), the set fis either a ray or a segment. Let X”’ be the 
vertex of [closest to X‘ Then, for sufficiently small e>0, the point 


X=X" +e (X"—X') EF. 

Hence 
wo & , cr 

Se Tipe 
and X’, X€M. Thus, X” is not a vertex of M, which contradicts (c) of the 
theorem. 

Thus, f=, and, by assumption, f=M, is one-dimensional. According 

to the second definition of an edge of a polyhedral set, [ is an edge of M. 
This completes the proof. 


§ 2. Representation theorem for a convex polyhedral set 


2-1, We have shown that any point of a bounded closed convex set is a 
convex linear combination of extremum points of this set (the representa- 
tion theorem), Applying this result to a bounded polyhedral set (a convex 
polyhedron) and noting that the number of its extremum points, i.e., ver- 
tices, is finite, we arrive at the following proposition. 

Theorem 2,1. An arbitrary convex polyhedron defined by restraints (1.1), 
(1.2) is a set of all points X of the form 


N 
X=% a, X;, (2.1) 
where X,, t=1, 2,.... Nare the vertices of the polyhedron, 


N 
Da,=1, a, >0. 


tal 

The representation theorem thus specifies entirely the structure ofa 
convex polyhedron. However, the proposition of the theorem is a priori not 
true for any unbounded set. Indeed, let M be an unbounded polyhedral set. 
Since M has a finite number of vertices, the set of points which can be 
writteninthe form (2.1) is bounded. Hence, it cannot coincide with M. 
Equation (2,1) is thus unsuitable as a representation of unbounded polyhedral 
sets. In this section we shall establish a representation theorem for an 
arbitrary convex polyhedral set. 

2-2. First assume that the rank of the system (1.1), (1.2) defining the 
polyhedral set M is a (equal tothe number of components of the vector X). 
This case is particularly important, since, as we shall show in § 4, any 
linear-programming problem can be reduced to an equivalent problem 
whose polyhedral restraint set has this property. 
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Theorem 2.2. Let the rank of the consistent system (1.1), (1.2) be n. 
In this case the convex polyhedral set mis a setofallpoints x of the form 


N, N, 
x= 2; X; + PAT R;. (2.2) 


Here X,, t=1,2,..., N, are the vertices of M; R,, i=1,2,...,N,@are the direction 
vectors of the unbounded edges of M; all the a,, B; are nonnegative; 

N, 

> a= 1. 

é=1 

Proof. We shall prove this theorem in two stages. Inthe first stage we 
shall establish that any X€M may be written in the form (2.2). This part 
of the proof is carried out by induction over the dimensionality @ of the 
polyhedral set M. In the second stage we shall show that any point X which 
can be written in the form (2.2) belongs to M. 

Stage 1, 1. Assume that g=1. The polyhedral set M is actually its own 
edge of maximum dimensionality. Hence, M, being a one-dimensional edge, 
is either a line, ora ray, or a segment (Theorem 1.5). The first possi- 
bility must be neglected since assuming the opposite would imply that the 
rank of the system defining Mis n—1. Thus, Mis either a ray ora seg- 
ment, If the former is true, the polyhedral set M can be written in the form 


X= X,+8,R,, B, = 0, 


where X, is the end point of the ray M, and R, is the direction vector of this 
ray. If the latter is true, M has the form 


X=a,X,+a,X,, a, 20, a, 20, a,+a,—1, 


where X,, X, are the end points of segment M. This completes the proof of 
the theorem for e=1 

2. Assuming now that the first part of Theorem 2.2 (which asserts that 
any X€M can be written inthe form (2.2))is validfor e<k—1, we shall 
prove its validity for g=&. Since the theorem has been proved for e=1, 
we shall assume k=>2. Without loss of generality we may assume that all 
inequalities (1.1) impose nonrigid constraints on M, and that the vectors 
D,, i=s+1,..., s+tare linearly independent. Then, according to Theorem 1.2, 
k+ t=a. 

By assumption, there are a linearly independent vectors among D,, i= 
=1,..., st#. Hence, there exist & vectors satisfying the nonrigid constraints 
of M which, together with the vectors D,,,,.... D,,, satisfying the rigid 
constraints of M, constitute a linearly independent system. Since k>2, 
there exist two vectors D, and D,, (1<i,, i,.<s) so that the system D,, D,, 
Dy +++, Dey, is linearly independent, 

3. Consider a point X,€M. First assume that this point reduces all the 
nonrigid constraints imposed on M to strict inequalities, i.e. , 


(D;, X,) <= di, i= be2ieves Ss. (2.3) 
Let Y=(j,, Yar. Yn) be a solution of the linear equations 
D ’ = — | a, A= 1,2; 
(Di. Y=(—)) ! (2.4) 
(D,, Y)=0., i=s-+].,..., s+ft. 


Observe that solvability of (2.4) follows from the linear independence of 
the vectors Dis D;.; Dy ayes D..3: 
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Let 
X (6) = X, + 46Y. 


By definition, the vector X(§) satisfies (1.2) for any 8. We shall show that 
this vector satisfies (1.1) only when 


a b<6’, (2.5) 
where = coo<@<0, 0<6’<oo. Indeed, 
(Dj, X(6))=(D;, X,)+6(D;, Y). 
The inequality (D;, X(#))<d, is therefore satisfied only when 
a if (Dn Y)>0, 
> y if (D,Y) <0. 


Hence, X(6) satisfies all the conditions (1.1) if and only if (2.5) holds for 
d;—(D,, X,) 


Dy Y9<0 (Dj, ¥) 
) di—(D,, X,) 
6°= mi 
ager (Dj, Y) 
(i= ee aes S). 


According to the definition of the vector Y, 


(D,, Y)<0, (O,,, Y)>0 
Therefore -—co < 6’, 0” < oo. 
It follows from (2.3) that 0’<0, 6’>0. Consider the points x’=X(6’), 
XxX” =X (6). Obviously, 


X, = 8X +8", (2.6) 


where i= 6. = a>. The expressions for 6, and 6, show that 6,>0, 
6,>0, 6,+6,=1, i.e., the point X, is a convex linear combination of the 
points xX’ and X. 

Depending on the choice of the numbers §’ and 6’, there exist constraints 


of M with subscripts i’ and ?/ so that 
(Dy, X)= dy, (Di, X") = dy. 

Hence, it follows that x’ (X”’) belongs to the face M’(M’) of the polyhedral set 
M whose restraint system is obtained from (1.1), (1.2) when the inequalities 
i’ (’) are replaced by equalities. Since (Dj, Y) #0, (Dy, Y) #0, the systems of 
vectors 

Dy, Dei ray Dyits 

Dy, Dsyy le D, 4+ 


are linearly independent. The dimensionalities of the faces M’ and M’ are 
therefore not greater than 
n—(t+1)=k—1, 


Applying the assumption of induction, we see that the points X’€M’ and 
x”E€M’ have the form 


’ 


N, N, 
x = LaXi+ DBR, (2.7) 
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N, 


N 
X = Laux: + VBR. (2.8) 


Here X;(X;) are vertices of the polyhedral set M’(M’): R,(R)) are the direction 
vectors of the unbounded edges M’ (Mw. All the oj, a, B;, B; are nonnegative; 


My, Ni 
ao = 1, Da;=1. 
=i dust 
4. We shall now show that the points x;, X; are vertices of the polyhedral 
set M, and R; and R; are the direction vectors of the unbounded edges of 


this set. We observe that if M, is a face of the polyhedral set M,, which in 
turn is a face of the polyhedral set M, then M, is a face of M. Indeed, let 


X=, X,+ VA (2.9) 


where XEM,; X,X,EM y,+7,=1) yp ¥,20. Since M,cM, we have XEM,. 
Hence, according to the first definition of a face, we observe that X,, X,E€M,. 
Applying this definition to the face M, of the polyhedral set M, we obtain the 
desired inclusion, X,, X,€ M,. 

Thus, a point X€M, has the form (2,9) only if the points X,, X, entering 
the representation also belong to M,. 

Applying again the first definition of a face, we conclude that M,is a 
face of M. Hence, in particular, follows the previous proposition regarding 
the points X;, X; and the vectors Rj, R;. 

Substituting in (2.6) X’ and X”as expressed by (2.7) and (2.8), respecti- 
vely, we obtain X,as a sum of a convex linear combination of the points X; 
and X; anda nonnegative linear combination of the vectors R, and R;. Since 
X;, X; are vertices of M, and R;, R;are direction vectors of the unbounded 
edges of M, we conclude that any point X, satisfying the inequalities (2.3), 
can be written in the form (2.2). 

5. Suppose that X,€ M does not satisfy (2.3), i.e., there exists an r-th 
nonrigid constraint of M(t<r<s) so that 


(D,, X)=d,. (2.10) 


In this case X, belongs to the polyhedral set M, defined by (1.1) with i#r, 
(1.2), and the equality (2.10). 
Observe that the system of vectors 


Dy, Dsayy Dsaar veer Daas (2.11) 


is linearly independent. Since by assumption the r-th inequality of the 
system (1.1) is a nonrigid constraint of M, there exists a point X°€Mso that 


(D,, X’)=d <d,. 
Let 
Y= X,—X’. 
The vector Y satisfies the equations 


(D,, Y)=0, i=s+l, oeey s+f, S 


(D,, Y)=d,—d>0. (2.12) 


Relationships (2.12) indicate that D, cannot be represented as a linear com - 


bination of the vectors D,,,,..., D,,, Further, noting that the vectors satis- 
fying the rigid constraints of Mare linearly independent, we confirm the 
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linear independence of the vectors (2.11). Thus, the rank of the system of 
rigid constraints imposed on the polyhedral set M, is not less than t+ 1 and, 
consequently, the dimensionality of this set is at most 


n—(t+1)=k—1, 


As in point 3 of the proof, applying the assumption of induction we verify 
that X can be represented in the form (2.2). 

We have thus shown by induction that any point of a polyhedral set M, 
whose dimensionality @ is equal to k=>2, can be written in the form (2.2). 
Since the theorem applies for o=1, this completes the proof of the first part 
of the theorem. 

Stage 2. 6. It now remains to show that any point, which can be written 
in the form of (2.2), belongs to the polyhedral set M. 

We first note that N,>1. Otherwise, (2.2) would contain only the vec- 
tors R; The set of all these vectors is nonempty, since, by assumption, 
there exists a point X€M which, according to the first part of the theorem, 
can be written in the form (2.2). Hence, M has unbounded edges each of 
which, inthiscase, is a ray (by assumption, the rank of the system (1.1), 
(1.2) is 2), The end point of any such ray is a vertex of M. Therefore, 
contrary to our assumption, M has vertices. Thus, a polyhedral set M al- 
ways has extremum points. 

7. Let R be the direction vector of any unbounded edge of the polyhedral 
set M. 

In our analysis of unbounded edges of polyhedral sets (§1) we have shown 
that 

(D, R)<=0, i=l, 2,..., 5, (2.13) 
(D,, R)=0, fd=st+l,..., s+#. (2.14) 


Consider now a point X, of the form 


N Ms 
x,= > a,X,+ 2 BiRi 


imi 


N 
where all a,, B; are nonnegative and So 1. We shall show that X,€M.In fact, 
{=z1 


Ni ; N 
(D;, X,)= 2 a,(D;, X,)+ 2 B,(D;, R,)- 


Applying the inequalities (1.1), which the points X, satisfy, and the inequali- 
ties (2.13), which hold for each vector R,, we obtain for lai<s 


N 
(D, X,)<d; Da;=d, 


f=1 
With s+1<a, ia s+, we obtain from (1.2) and (2.1 4) 
N 
(D;, X)=d, Yay=d, 
Point X, thus satisfies all the conditions (1.1), (1.2), i.e., it belongs to 
the polyhedral set M. This completes the proof of the second part of the 
theorem, 
Theorem 2.2 contains, as a particular case, the result of Theorem 2.1, 
Indeed, if M is a convex polyhedron, M has no unbounded edges, i.e. , N,=0. 
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In this case the representation (2.2) reduces to the representation (2.1) of 
Theorem 2.1. 

In our proof of Theorem 2, 2 (point 6) we established an important pro- 
position, which may be summed up in the following theorem. 


Theorem 2.3. If the rank of the restraint system (1.1), (1.2) defining a 
polyhedral set Mis n, M has at least one vertex. 

According to Theorem 1. 4.to each vertex of a polyhedral set Mcorres- 
pond a linearly independent restraints from (1.1), (1.2), which are reduced 
to equalities at this point. Therefore, the existence of at least one vertex 
of M shows that the rank of the system (1.1), (1.2) is 2. Combining this 
result with Theorem 2.3, we arrive at the following result: 


Theorem 2.4. For a nonempty polyhedral set M to have ai least one 
vertex, itis necessary and sufficient that the rank of the restraint system 
(1.1), (1.2) be n. 

2-3. Theorem 2.2 establishes the structure of a polyhedral set M for the 
case when the rank of the system (1.1), (1.2) is exactly a (the dimensionality 
of the space of points X). Let us now investigate the general case, when 
the rank of the system (1.1), (1.2) is rea. 

Let the rank of the system of vectors D,,i=1,2,...,s+t, be r<a. This 
indicates that the matrix ||d,,|l,,;,, has r linearly independent columns, the 
remaining a—r columns being linear combinations of the other columns. 

Let the j-th column of the matrix ||d;,|l,4;,,,, be denoted by 


DY = (dij, day, eos Gyae, )?, = f=1, 2,200, (2.15) 


The restraint system (1.1), (1.2) can now be written in the form 
SL nil 
) 

2 x,D 


Here D=(d,, d,, ..., d,,;)7; the sign = indicates that the first s components 
of the vector relationship (2.16) are inequalities, and the subsequent # com- 
ponents are equalities. 

By assumption, the maximum linearly independent subsystem of the 


IIA 


D. (2.16) 


system DY, j=1, 2,..., acomprises r vectors. Without loss of generality 
we may take these to be the first r vectors of the system (2.15) 
DO De oe DY: (2.17) 


We now express the vectors D” with j>r as linear combinations of the 
vectors (2.17) 


DY = & 8,,D". (2.18) 


Applying the equality (2.18), we reduce the restraint system (2.16) to the 
equivalent form 


p> (i+ 2 x5) D° SD. (2,19) 
=rt+i 


{=1 


Consider the polyhedral set M, whose restraint system comprises (2.16), 
or (2.19), and the equations 


Ce ee eI (2.20) 


We easily observe that the rank of the restraint system of the polyhedral 
set M, equals exactly a. 
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We now establish a relationship between the polyhedral sets M and M,, 
which will enable us to apply the previous results to the analysis of the 
structure of M. 

Let II,_, be an (n—r)-dimensional subspace generated by the intersection 
of r, (n—1)-dimensional subspaces (hyperplanes) defined by linearly inde- 
pendent homogeneous equations 


Ly (X=, >> 6,,x,=0, t=1, 2,004 (2.21) 
+i 


Lemma 2.1. A polyhedral set Mm is the sum* of the polyhedral set M,and 
the subspace I1,.-,: 


M=M,+Il,-; 
Proof. 1. Let X’€M,, X"Ell,_,. We shall show that in this case 
X=X'+4+XEM. 


Representing the restraints defining Minthe form (2.19), we obtain 
240% = DY (x; + > 8x) 08 = De (X) DO = YL (XD + YL (XD. 
é=1 4=1 


By assumption, ; 
DL (xX) DED (X" EM); 


(=a 
L (X=... HL, (X)=0 (XEN, _,), 
whence 


>> xy DYED, 

i.e, ,XEM. = 

2. Let now Xbea point in M. Let 
: L,(X) for j=l, 2,...7 
oo a 0 for j=r+1,..., a 

x) =X, —Xiy b= 1, 2 een 
Applying (2.19), we readily verify that 
X!' = (x', Xi, eeey x, EM,. 
From the definition of the points X’ and X"=(x', x1,..., #7) we have 
L,(X") = Lb, (X) —L, (X") = Lh, (X) — x, = 
= L,(X)—L,(X)=0, t=1, 2,..., 7 
Hence 
xell,-, 

Any point X€M can, thus, be represented as the sum of points xX’ €M, and 
X€II,., This completes the proof, 

In particular, it follows from Lemma 2.1 that the equality r=a”is a ne- 
cessary condition for the boundedness of a polyhedral set M. 

Let us now establish a correspondence between the faces of the poly- 
hedral sets M and M,,. 

We shall say that the faces TEM and T,€M, are corresponding 
faces if the restraints defining I, differ from the restraint system of the 
face I only by the additional equalities (2.20). 


* By definition, a set A is the sum of sets B and C (A=B+-(C), if XEA canbewrittenas X=y-+Z, where 
YEB, ZEC. 
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According to the Lemma 2.1, the face Tis the sum of the face IF, and 
the (a—r)-dimensional space I[],_, It can easily be verified (see Exercise 
4) that the dimensionalities @(f) and e(I,)of the faces [T and I,, respectively, 
are related by the expression 


e(F)=e(l,)+2—r, (2.22) 


where r is the rank of the restraint system defining the set M. Applying 
Theorem 2.4 to the polyhedral set M, and using equality (2.22), we arrive 
at the following proposition, 


Theorem 2.5. (Generalization of Theorem 2.4.) The minimum dimen- 


stonality of the faces of a polyhedral set M is n—r, where ris the rank of 
the restraint system (1.1), (1.2). 


We note that according to Lemma 2.1 an arbitrary face of minimum di- 
mensionality in M coincides with the intersection of the hyperplanes 


Rn 
° =X gry i= 1, 2, every Fy 2.23 
x; + 2 8; /%; at 2 6; /z; r ( ) 


where X=(3,, X,,...,%,) is some vertex of the set M,. 

Lemma 2.1 makes it possible to extend the principal theorem of this 
section, Theorem 2.2, to the case of an arbitrary polyhedral set. 

Theorem 2.6. (Representation theorem of polyhedral sets.) Anarbitrary 
polyhedral set Misa set ofall points X of the form 


N, N. nr P 
x= aX;+ % BRit DR (2.24) 
=1 =1 s=1 
N, 
where >» a,;=1; a, 20, B;=0; X; arethe vertices of M,; R; ave the direction 


peCtors: op the unbounded edges of M,; Ri, i= 1, 2,..., n—r are any complete 
system of linearly independent vectors of the subspace Il,_,; ris the rank 
of the restraint system defining M. 

Theorem 2.6 is an obvious consequence of Theorem 2.2 and Lemma 2,1. 


2-4. To illustrate the results of this section, we consider two numerical examples, 
Example 1. Consider a polyhedral set M, defined by the restraints 
—x,—%x, <9, 
—xX, +4, <0, 
—X, <= 2, 
—%X, + 2x, <& 2. 
The image of M, inthe +x,0x,-plane is shown in Figure 2.1. We observe from Figure 2,1 that the faces of 
minimum dimensionality of M, are the points X,=(2,— 2), X,=(2, 2), and X,=(0, 0). This conculsion 
is in agreement with Theorem 2,4, since the rank of the system (2.25) is n=2 (the vectors D,=(—1, —1), 
D,=(—1, 1) are linearly independent). The polyhedral set M, is not bounded and, consequently, it should 
have unbounded edges, We observe from Figure 2.1 that this set has two unbounded edges issuing from points 
X, and X, and having R,=(1, 0) and R,=(2, 1) as their direction vectors. 
According to Theorem 2,2, the polyhedral set M,can be represented as the sum of an arbitrary convex 
combination of the points X,, X,, X, and an arbitrary nonnegative combination of the vectors R, and R;, 
e., M, is a set of all the points 


(2.25) 


X=a, (2, — 2)-+a, (2, 2)+4, (0, 0) +B, (1, 0) +6, (2, 1), 
where a,;>>0, B;>0, a,+0,+0,=1, OF 


X = (2a, + 2a, + B, + 26,, — 2a, + 2a, + B,), 


where a,>0, a,2>0, B,=>0, B,2>0, a, taal. 
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Example 2. Consider a polyhedral set M defined by the restraints 


—X,— x, — 2x, = 0, 
—*%, +; <0, (2. 26) 
—X, —x,< 2, 
—X,+2x,+x%, & 2. 


Here n=3; it is, however, easily verified that the rank of (2,26) is 2. Let 
DY = (— l, —I, 0, —1)', 
D®=(—1, 1, —1, 2)7, 
D® == (—2, 0, —1, 1)7. 


The vectors D™ and D®) are obviously linearly independent, whereas D@) =D + D®), 


2, Ss Pre. 
OY Wy 

x, , ‘ 

ee" 
4 ( : wo 
i, 
k, 
“e 
‘o 
FIGURE 2,1 FIGURE 2,2 


According to the general rule, the polyhedral set M, is defined by the restraints (2.26) and the additional 
equality x,=0. 
We observe that M, coincides with the polyhedral set M, discussed in the preceding example. Since n—r=l 
here, the subspace II,_,=II, is a line, Transforming (2,26) to the form 

— (*, + *,)— (4, + %,) SO, 
— (4, +45) + (Xs +%) SO, 
—(% +4) <2, 

— (x, + x) — 2(x, +%;) <2, 


we observe that this line is defined by 
X,+x,=0, x,-+2x,=0. (2,27) 


According to Lemma 2,1 the polyhedral set M can be considered as consisting of all the lines parallel 
to (2.27), and passing through points of M,. The image of M is given in Figure 2,2, 

Since n—r=1, the minimum dimensionality of the faces of Mis 1, The polyhedral set M has three 
faces of minimum dimensionality: these are X,X|, X,X,, and X,X,. Each of these lines corresponds to a 
definite vertex of M,. Applying formula (2.24), we give the general representation of the points XE M. 

In this case M,=M,. Therefore, X,=(2, — 2, 0), X,=(2, 2,0), X,=(0, 0, 0), R,=(1, 0, 0), R,=(2, 1, 0). 
The vector R (the only one, since an—r=.1) is the direction vector of the line (2,27) 


R,=(—1, —1, 1). 
Thus, the polyhedral set M is a set of all the points X of the form 
X=a,X, +a,X,+4,X,+B,R, +£,R,+ viR, 
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or 


X = (2a, + 2a, +B, + 2B.—¥y, — 20, + 2, + B— V1. Vi)» 


where 


a,220, a,2>0, 6,290, B, 20, a,+a,<1. 


§ 3. Equivalence of the two definitions of a 
convex polyhedral set 


3-1. Theorem 2.2, establishing representation (2.2) for any polyhedral 
set with r=n, can be stated in alternative form: 


Any polyhedral set M with r=n can be written as a sum of a convex 
combination of the vertices of M and a nonnegative linear combination of 
the direction vectors of the unbounded edges of M. 

It is easily observed that any s-dimensional subspace is a nonnegative 
linear combination of a finite number of points. Indeed, if X,, X,,..., X, is 
an arbitrary linearly independent system of vectors in this subspace, the 
desired system may be taken as comprising all the points of the form 


Xi Xe Pesy XxX; —X,, —X;,, eos — X,. 


This system of points consists of 2s elements. It can be shown that the 
smallest system having the same properties contains only s+1 points 
(Exercise 5). We can now advance a new formulation of the general repre- 
sentation theorem for a polyhedral set (Theorem 2.6), analogous to the 
alternative formulation of Theorem 2.2. 

Any polyhedral set can be written as a sum of convex and nonnegative 
linear combinations of some systems of points. In this section we shall 
establish the converse proposition; this will introduce a new definition ofa 
polyhedra! set which is useful in some theoretical problems of linear pro- 
gramming. 

3-2, We consider a certain class of convex polyhedral sets, significant 
in the following (for the definition of a convex cone in any finite-dimen- 
sional space, see Appendix, 3-2). 

A polyhedral convex cone is a convex polyhedral set having the 
properties of a cone. In other words a polyhedral set M defined by the re- 
straints (1.1), (1.2) is a polyhedral cone with apex at X,(the adjective ''con- 
vex'' is omitted for the sake of brevity) if from X€M it follows that 


x, th(X—X,)€ M 
for any A>0. 
Let M be a polyhedral cone with apex at X,and let X@M. Then, for any 
i, isiggs, 
(D;, X—X,) = 0. 
Otherwise there exist 4>0 such that for some i 
(Dj, X, +A(X—X,)) > dy. 


This inequality is not in accordance with the property of the cone M 


X,tA(X—X,)EM forany As. 


ol 


Since X,E€M, 
(D,, X— X,) = 0 


fors<i<s+t Thus, any X€M satisfies the restraints 
(D;, X) <= (D;, X,)s i=1, 2; eo og s, (3.1) 
(D;, X) = (Dj, X,)s i=stl, ea ayg s+ft. (3.2) 


An arbitrary point obeying (3.1), (3.2), obviously satisfies (1.1), (1.2) 
and as such is contained in a polyhedral cone M. 

The restraints defining a polyhedral cone may thus be reduced to the 
form (3.1), (3.2). It can easily be shown that any set defined by the re- 
straints (3.1), (3.2) is a polyhedral cone. 

Let K be the polyhedral cone defined by the restraints (3.1), (3.2). Since 
the apex X,of the cone K satisfies the equations 


(D,, X)=d,=(D,, X,), 1.2... , HF, 
the vector D=(d,, d,, ..., d,,,)7is a linear combination of the vectors 
DY, j=1, 2,...,.0. 
The rank of the system of vectors 
D, DY, j=1, 2....,5 4, 
thus equals the rank of the subsystem 
DY, j=1,2, 0... 0. 
Further, noting that the ranks of the systems 


Du i=l, 2; eees s+; Di = (dis eee Giny d,), 
f=1,2,..., s+, 


are equal, respectively, to the ranks of the systems 
D, j=1,2,..., a D, D”, j=1,2,...,2, 


we conclude that the vector systems 


D,, i=1,2,..., 2 and D, i=1,2,...,2 
have the same rank. Therefore, if D;,..., D;, is the largest linearly inde- 
pendent subsystem of the vectors D,, i=1,2,..., s+# the vectors D,,,..., Dy, 


constitute the largest linearly independent subsystem of the vector system 
D,, i=1, 2, ..., SEL 
Hence, the point X satisfying the system of equations 
(D,, XY=d;,, a=1, 2,...,7; (3.3) 


where D,;,,a=1,2,...,ris the largest linearly independent subsystem of the 
system of vectors D,, i=1,2, ..., s+#, reduces all the restraints (3.1), (3.2) 
to equalities. 

We can now easily establish the uniqueness of the minimum face of the 
polyhedral cone kK. If X is a point belonging to some minimum face of the 
cone K, whose system of defining restraints isof rank r, it should satisfy 
equations of the form (3.3) (Theorem 2.5) and as such reduce all the re- 
straints (3.1), (3.2) to equalities. Hence, any minimum face of the cone K 
coincides with the set of solutions of the system (D,, X)=d,, t=1,2,..., s+#. 
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Therefore, the polyhedral cone K has but one face of minimum dimensiona - 
lity. According to Theorem 2,5 the minimum face of the cone K is a—r- 
dimensional, where / is the rank of the restraint system (3.1), (3.2). In 
particular, for r=a, the point X,, the apex of the cone K, is the only ex- 
tremum point of K. In this case xX, is generally termed the point of the 
cone K., 

Observe that, in general, the apex of a polyhedral cone is not equivalent 
to a vertex of a polyhedral set. The two concepts coincide only if the apex 
of the cone is also the point of the cone. Otherwise, any point belonging to 
the minimum face of the cone can be taken as its apex. Obviously, none of 
these points satisfies the definition of a vertex of the polyhedral set repre- 
senting the cone, 

This inconsistency in concepts is due to the fact that terminology used 
today for cones and polyhedral sets developed independently. However, no 
confusion should arise. 

The apex of a cone will be taken as an arbitrary point belonging to the 
face of minimum dimensionality. 

Applying Theorem 2.6, we may assert that an arbitrary convex polyhe- 
dral cone K comprises all the points X of the form 


N 
X=X,+ 2 B, R,, (3.4) 


where f, >0, i=1,2,..., Ny; X,is the apex of K; R, is a set of vectors. 
3-3. Consider the set 7 comprising the points X written as 


N N, 
X= Ba, Xi+ 3 BR (3.5) 


Ny 
where a,=0, B;=0, > a,=1. The primary purpose of this section is to prove 
i=! 


that 7 is a convex polyhedral set. To this end we first establish three 
lemmas. 
Lemma 3.1. Let the vectors A,, A,,..., A, be linearly independent and let 


k 
2 a, A; |< Cy. 
= t 


There exists a number c, independent of the coefficients a;so that 


k 
p) la;[<o¢,. 


Proof. Suppose the contrary is true, i.e., let there exist a sequence of 


vectors 
(s 
al?) = (a, as”, eee a,” , 


so that . 
Pts Alec, (3.6) 
and, moreover, k 
lim » jaf?| =o. (3.7) 
srooliz=l 
Let 
ai) a al), 


From (3.6) and (3.7) we have 
r 
lim | > ai?4,|=0. (3.8) 
From the definition of a” we have 


> jal} == 1 (3.9) 


t=! 
for any s. Therefore, from the sequence {ato} we may choose a convergent 


subsequence {a}, ee 
lima? =a, ¢=1,2, ...) & 
{+a 


Hence, applying (3.8), we have 


Since the vectors A,, A, ..., 4,are linearly independent, 


a, mM, Eee =a,=0. 


However, from (3.9) 
k 
> [a;|=1. 
i=! 


This contradiction proves the lemma. 

Lemma 3.2. For any point X, which can be written in the form (3.5), 
there exist linearly independent vectors R,,, Ri, ..., Ri, of the system R,, i=1, 
2,..., N,, So that 

Ni: 


é _ 
X= 2 aXi+ 2 B, Ris (3.10) 


‘= 
M 
where a,>0, 6, >9, 2 a;=1. 
i= 
Proof. Consider the representation (3.5) of the point X with the minimum 
number of positive coefficients B;. Let (3.10) be one of these representa - 
tions. We shall show that in this case the vectors R,,, Ri, ..., Rj, are linearly 


independent. 
Suppose the vectors are linearly dependent, then 


i 
2 Ri =0, (3.11) 


where not all #, are zero. Without loss of generality we assume that one 

of the 4, >0 (if this requirement is not satisfied, we may multiply both sides 
of (3.11) by -1). Multiplying (3.11) by 6>0 and subtracting the result from 
(3.10), we obtain 


Ny a 
x= z a; X+ % — 64,) Ri. 


If we now take 


(= min Ps 
t >of 


we obtain a new representation of the point X with the number of positive 
B; not exceeding 


l-1 <1. 
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This contradiction shows that the assumption of dependence, equality (3.11), 
is false; the system of vectors 


Ri, R,,, coey Ri, 


is linearly independent. This completes the proof. 
Lemma 3.3. The set T of points xX, which can be written in the form 
(3.5), ts closed. 


Proof. Consider an arbitrary convergent sequence of points X“€T. To 
prove the lemma, we must show that 


lim X) = XO) ET, (3.12) 
t+o@ 


Since X™ is an element of T for any f, 


N, le 
Xt) 2s a af” Kit DBa.t Ri, 


where a} s0, Bae > 0, > a? = 

According to Lemrna 3.2, i. system of vectors Ri, ,, A=1, 2,..., 4, com- 
prising some of the vectors R,, i=1,2,..., N, is linearly independent for 
any f. 

The number of points X“ is infinite and the number of possible systems 
which can be constructed from the vectors R,, R,,..-, Rw, is a priori finite. 
Therefore, there exists an infinite subsequence{ X*)} of the sequence {X(} 
so that 


Ny ! 
Xt) = Ya X + SF pY Ri, (3.13) 
{=1 A=! 
where R,,, Ri, ..., R;, is a linearly independent By ere of vectors, which is 
the same for all the points Xr), a) >0, PY > 0, by a} = 
Since the subsequence {Xe}, converges, (ieee gists a number ce, so that 


| X(fr) | <= c, 


forany r. Therefore, applying the principal properties of vector norms 
(lengths), we obtain for any r 


é N; 
N N 
+|3 aj” x; | — C, +(3 of) C.=C, +6, =6,, (3.14) 
where 
¢,=ma ll 
lqic 


According to Lemma 3.1, we obtain from the inequality (3.14) a constant 
¢, independent of r so that 


<C,. 


Thus, in (3.14), not only the numbers a” (0<a <1)are bounded, but also 
the coefficients pf (0o<pY’ <c,). This proves the existence of convergent 


subsequences of the sequences {a}, dant, 2,00. Ms (PV, AI, 2, 00, 
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Hence, there exists a subsequence of indices r,,7,,...,ry,..-, so that 


Lig oe al”), $2122), aca dN 
tim BX y= BY”, atl (ay Sepereans 
Obviously, 
a) 0, i= 1,2,..., M3 B00, A=1,2,..., l; 


N, 
> af? = 1. 


i=) 
Now, if in (3.13) r tends to infinity over the subsequence ry, we obtain 


Ny t 
ZX + DBR, =X. 


Hence, applying the properties of the coefficients a, Bf, we obtain (3.12), 
which proves the lemma. 

3-4, We now formulate and prove the converse representation theorem 
for a convex polyhedral set. 

Theorem 3.1. A set T comprising points x, representable in the form 
(3.5), is a convex polyhedral set. 


Proof. 1. Consider a polyhedral set K of points 
Y=(¥, Vi=(yys Yar eo Yar Vds 


defined by the restraints 


(Y, X)—-V<o, be le 2) isc IVa, (3.15) 
(Y, R;) <0, i=, 2. eeey N,. 
It follows from (3.15) that K is a polyhedral cone with apex at the point 
O=(0, 0,..., 0. Therefore (see (3.4)), there exist points Y,, Y,,..., Yy such 
that the cone K can be written as 
_ N  — ON 
Y=(Y, VYj= z Vv; i= Dvwl¥o Vi), V7; = 0. (3.16) 
With the points Y,, Y,,..., Yy we form a polyhedral set T, of points X whose 
defining restraints have the form 
(Yn XV, i=1,2,..., N. (3.17) 


2. We shall show that the polyhedral set T, is identical with the given 
set T. Consider an arbitrary point X€T. By assumption, 


Ny: N; 
X= 2axit B Bi Re 
N, 
where a,>0,f;=0, Dia,=1. Hence, for any i(l<i<Q), 
f=1 


N, N; Ni, 
(Y;, X= 2 (Yp X)+ 2 B(Y;, R= V;, 2 UH Vi (3.18) 


We have used the inequalities (3.15), which apply for the points Y,;, and the 
properties of the coefficients a,, By. 
Inequalities (3.18) indicate that XE€T,. We have thus proved that 


aromas ae 


3. Let us now establish the inverse. Consider a point X,€T, and assume 
that 


X,¢T. 
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According to Lemma 3.3, T is a closed set. The convexity of T is obvious. 
Hence, the hyperplane separation theorem (see Appendix, Theorem 3.1) 
applies to this set and to the point X,¢T. According to this theorem there 
exists a vector A, and a number e¢, so that 

(A, X,)>e, (A, ee, (3.19) 


for all XET. Since (3.20) holds, in particular, for all X,, the point A, = 
=(A,, ¢,) satisfies the first N, restraints of the system (3.15). 
Let xX’ be some point in T. In this case, for any iand B=0 the point 


X,= X' + BR, ET. 
Hence, 
(A,, X,) = (A,, X’) + B(A,, R)<c, 
for any B=>0. This is possible only if 
(A,, R,) <0. 


The point A,=(A,, ¢,) therefore satisfies the last N, restraints of (3.15). 
Thus, A,€X. Hence, applying (3.16), we have 
N 


A= ve Yn 
i= 


N 
oo p> Ye Vin Y = 0. 
By assumption, X, satisfies (3.17) (X,€T,). Hence 


N N 
(A,, X,) = a (Y,, X)< 2 yy” Vi = ¢,. 


This inequality contradicts (3.19). The assumption that X,4¢T is therefore 
false. Thus, any point X, of T, belongs to T, i.e., 


Baad Is 
From this relationship and that established at the end of point 2, we have 
T=T,. 


This completes the proof. 
3-5, For the sake of brevity, we say that the set T comprising points X 
of the form (3.5) is spanned (or generated) by the points X,,X,,..., Xv,and 


the vectors R,,R,,..., Ru,. 
We remind the reader that points and vectors are, conceptually, equi- 
valent. Referring to X,,..., Xv, as points and to R,,R,,..., Rv, as vectors we 


only emphasize the difference in the roles of these elements in spanning T; 
the set T coincides with the sum of convex combinations of the points 

X,, X,, -.-, Xv, and nonnegative linear combinations of the vectors 

R,, R,, .... Rv, AS we have already indicated, any polyhedral set is spanned 
by a finite number of points and vectors. On the other hand, by Theorem 
3.1, proved above, any set spanned by a finite number of points and vectors 
is a polyhedral set. Thus, we can introduce an alternative definition of a 
convex polyhedral set. 

A convex polyhedral set is a set spanned by a finite number of 
points and vectors (a set which is the sum of convex combinations of the 
given points, and nonnegative linear combinations of the given vectors), We 
shall refer to this definition as the second definition of a polyhedral 
set. The definition given in §1, according to which a polyhedral set is the 
intersection ot a finite number of half-spaces and hyperplanes will be 
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referred to below as the first definition of a polyhedral set. The 
equivalence of these two definitions, established by Theorems 2.6 and 3.1, 
is one of the underlying concepts of the theory of convex polyhedral sets. 

Above we discussed two important classes of convex polyhedral sets, i.e. , 
the class of convex polyhedra and the class of convex polyhedral cones. 
According to the first definition, a convex polyhedron is a bounded set which 
is the intersection of a finite number of half-spaces and hyperplanes. Since 
a bounded set may not contain rays, the second definition of a convex poly- 
hedron is formulated as follows: a convex polyhedron is a set com- 
prising all the convex combinations of a finite number of points. 

We have seen that a polyhedral set is a convex polyhedral cone if and 
only if the restraint system of this set can be reduced to the form (3.1), (3.2). 
Therefore, according to the first definition of a polyhedral set, a convex 
polyhedral cone may be defined as the intersection of a finite number of 
hyperplanes and half-spaces, whose boundary hyperplanes intersect ata 
single point, called the apex of the cone. (We note that the boundary hyper- 
plane of a hyperplane is the hyperplane itself, ) 

Applying the second definition of a polyhedral set, we say that a convex 
polyhedral cone with apex at X,, consists of all nonnegative linear 
combinations of a finite number of vectors translated to the point X,. 


§ 4. Principal properties of the linear-programming 
problem 


4-1, Consider the general linear-programming problem written in arbi- 
trary form. 
Maximize the linear form 


n 
(C, x)= »» C,%) (4.1) 
subject to the conditions 
(D,, x)= »2 dj )X,<adiy i= |, 2, eoey §5 (4.2) 
j=i 
(Dj, X= % dj xj=d;, i=s+l,...,s+8. (4.3) 
: = 


We assume that the restraint system (4.2), (4.3) is consistent. In this 
case all the points X satisfying (4.2), (4.3) constitute a convex polyhedral 
set. Let this set be denoted by M. Problem (4.1)-(4.3) thus entails maxi- 
mizing a linear function (4.1) defined on a polyhedral set M. This set is 
sometimes called the domain of definition of the linear form, or the 
polyhedral set of the restraints of the given problem, 

In this section we indicate some important properties of the problem 
(4.1)-(4.3); in the process we shall draw upon the various aspects of the 
theory of polyhedral sets discussed in previous sections, 

We first observe that the rank of the restraint system (4.2), (4.3) (the 
rank of the system of vectors D,, i=1, 2,...,s+t)can always be taken to equal 
the dimensionality of the space of points X, i.e,, a. To verify this, as- 
sume that the rank of system of restraints (4.2), (4.3) is r<a. Applying 
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Lemma 2,1, we have the representation 


M=M,+T1 (4,4) 


| Saal 


where M, is a polyhedral set, whose defining system of restraints is of 
rank a, II,_, is an (n—r)-dimensional subspace. Two cases are possible: 
(a) a point Ye€II,_, exists so that 


(C, Y= 2B cyi=o #0; 

(b) (C, Y)=0 for any YETI, _,. 

It is easily seen that in case (a) the linear form (4.1) is unbounded from 
above on the set M(on the set of feasible programs of the problem), Indeed, 
since IIJ,_, is a subspace, AY€EIl,_, for any A. 

Let X,=X-+AY, where X is any point in M,. According to the represen- 
tation (4.4), X,€M for any 4. Consider 


(C, X,)=(C, N+A(C, YY=(C, XAT. 


Obviously, sup(C, X,)=oo for #0, i.e., the linear form of the problem is 
unbounded in its domain of definition. 

For case (b), consider the maximization of the linear form (4.1), sub- 
ject to the restraints defining the polyhedral set M,. If this problem is un- 
solvable, then according to (4.4) the problem (4.1)—(4.3) has no solution 
either. Let M* and M, denote the sets of optimal programs with the linear 
form (4.1) and restraint polyhedral sets M and M,, respectively. Applying 
(b), we obtain from (4.4) 


M* = M, +I, _,. 


Thus, to solve the problem (4.1)-(4.3), the polyhedral set M may be re- 
placed by M,. If the resulting problem is solvable and {b) is satisfied, 
then solution of this problem yields the optimal program of the initial prob- 
lem. Otherwise, the initial problem has no solution, 

The transition from M to M, entails calculating the maximum number of 
linearly independent vectors inthesystem D,, i=1, 2, ...,s+#t. The sub- 
space II,_, is obtained by representing the remaining D, in terms of the 
linearly independent subsystem (see 2-3), 

In this section we shall assume that these transformations have been 
carried out (if transformations were indeed necessary) so that the rank of 
the restraint system defining the polyhedral set of the problem is always 
taken to equal nz. The necessity of carrying out these transformations is 
generally established during the preliminary analysis of the problem (when 
the first approximation is being computed). This aspect of the problem is 
dealt with when encountered. 

4-2. When analyzing linear-programming problems, special significance 
is attached to programs corresponding to the vertices of the polyhedral set 
of restraints. (In the following the reader will often see the validity of this 
statement.) We shall, therefore, discuss separately the class of these 
programs. 

A program X=(x,, ..., *,) of the problem (4.1)-(4.3) is called a support 
program if there are a linearly independent restraints among the rela- 
tionships (4.2), (4.3), which are reduced by the program to equalities. 

It follows from this definition that the concept of a support program is 
equivalent to a vertex of the polyhedral set defined by the restraints (4.2), 


59 


(4.3) (see Theorem 1.4). The number of support programs of a linear- 
programming problem is, therefore, always finite. When the domain of 
definition of the linear form of the problem is bounded, i,e., a convex poly- 
hedron, any feasible program of the problem is a convex linear combina- 
tion of its support programs. 

Since the rank of the restraint system (4.2), (4.3) is 2, applying Theorem 
2.4, we obtain the following result. 


Theorem 4.1. (Existence theorem for support programs.) If the set of 
feasible programs of problem (4.1)-(4.3) 1s not empty, at least one of the 
feasible programs is a support program. 

We remind the reader that a solution of the linear-programming prob- 
lem is a program for which the linear form of the problem is conditionally 
maximized or minimized, depending onthe exact statement of the problem. 

A problem having at least one solution is said to be solvable. 

Let X,, X,, ..., Xv,consist of all the support programs of the problem 
(4.1)-(4.3); R, R,, ..., Rv, are the direction vectors of all the unbounded ed- 
ges of the polyhedral set M. In this case, according to Theorem 2.2, the 
polyhedral set of restraints of the problem (4.1)-(4.3) comprises points of 
the form X 

Mi 


N; 
X= 2 a,X,+ py BRi (4.5) 


where a; >0, B;>0, 3 o=1. 


Assume that the prebled (4.1)-(4.3) is solvable. Then it can be easily 
verified that for any i, 1=i<N,, 


(C, R;) <0. (4.6) 
Indeed, if for some i=i’ 
(C, Rv) =d>0, 
then, taking X(Bp)=X,+fR,, we obtain 
pina (C, X(B)) == oo, (4.7) 


But, for any B>0 the vector X(f)is a program of the problem (4.1)-(4.3). 
Hence, (4.7) indicates that this problem is unsolvable (the linear form of 
the problem is unbounded from above on the set of feasible programs). 
Thus, for solvable problems, (4.6) applies for any vector R,. 

Assume now that X* is a solution of the problem (4.1)-(4.3). Applying 
(4.5), which is valid for any feasible program of the problem, we have 


N, N, 
X* = x aX, + p>} BrRi (4.8) 
N, i 
a, 0, pf, =O, mu = 1. 


We now choose a subscript i=i’so that af, >0. We express xX; from (4.8): 


Xp =, [x*— > a@X,— x B:R,| : 


Gyr tei! (=1 


Taking the scalar product of both sides of this equality with the vector C, 
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we obtain 


iS 
x 
D 


Ny 
Se [(C, x— Dali, XI— DBC Ry] > 
7 


i a ée1 


N, 
Ss [(C, X*)-(C, x) Da —D) BG R)| > 
aj, ine (=s 
oe (tc. x*)-(c, x) > a; | = 
Qi) ize’ 
=(C, A) (1- Yh al) =(c, »). 
Gyr (eu 
The first inequality follows from the fact that X* solves the problem; the 
second inequality follows from (4.6); the last equality follows from the con- 
dition 


Thus, 
(C, Xp) = (C, X*), 


On the other hand, since X* is a solution of the problem we have 
(C, Xr) <a (C, X*). 
Comparing the last two inequalities, we obtain 
(C, Xj) = (C, X*), (4.9) 


where, by assumption, X, is a support program of the problem (4.1)—(4.3). 

A solution providing a support program of a linear-programming prob- 
lem will be called a support solution of this problem. The relation- 
ship (4.9) is equivalent to the following proposition, which is highly im- 
portant in linear programming. 

Theorem 4.2. (Existence theorem for support solutions.) Any solvable 
linear-programming problem with a restraint system of rank n has at least 
one support solution. 

4-3, Theorem 4.2 indicates the following method for solving linear-pro- 
gramming problems. Compute all the support programs of the problem. 
This can be done by investigating C}~‘systems of linear equations each con- 
taining ¢ equations (4.3), and a—t equations satisfying the conditions (4,2). 
Then, compute the value of the linear form (4.1) for each of the resulting 
support programs, whose number is finite. By Theorem 4.2, if the prob- 
lem is solvable, the support program corresponding tothe highest value of 
the linear form solves it. However, this method is impracticable if n—t 
and sS>n—t are fairly large (as is generally the case in applied problems). 
Let, for example, t=0, s=2a. Applying Stirling's formula, we have 


I 
Ch, = = 2". 
nS a, 


Investigating a linear equations in a unknowns (determining their solutions 
or detecting linear dependence in the given system of vectors) requires 
about zn’ operations. 

Hence, ee operations will be required to establish all the support 


programs of the problem. Assuming the problem to be programmed fora 
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computer performing 10° operations per second, the optimal program for 
n=25 will be found after approximately 10° years. 

Even for comparatively small problems (2=25), this method involves an 
astronomical number of computations and is obviously of no practical signi- 
ficance. Nevertheless, in spite of the unsuitability of the method based on 
evaluating all the vertices of the polyhedral restraint set, the concept of 
vertex evaluation is quite useful. All the methods (apart from the iterative 
methods) of linear programming entail, to a certain extent, evaluation of 
a sequence of vertices of some polyhedral set (not necessarily the poly- 
hedral set of restraints of the problem in question), This evaluation, how- 
ever, is carried out in such a way that only a small part of all the vertices 
available need be considered to solve the problem. An essential reduction 
in alternative programs to be considered is achieved with the aid of two 
properties which must be characteristic of any practicabie method of eva- 
luation: 

(a) ordered evaluation, i.e. , arranging the vertices so that a 'worse'' 
vertex is never preceded by a "better" vertex (the concepts of ''worse"’ and 
"better are inherent of the method); 

(b) availability of a criterion making it possible todecide, without evalua- 
ting all the vertices, that the ''best'’ vertex has been found (the vertex 
giving the solution of the problem), Each linear-programming problem has 
its characteristic method of ordered evaluation and criterion. 

4-4, If a linear-programming problem has a unique solution, then, by 
Theorem 4.2, this is a support solution. Assume that the problem has 
several solutions. Consider one of the solutions, say X*. The vector x* 
can be represented in the form (4.8). In deriving (4.9) we showed that if 
a; > 0,X; solves the problem (4.1)-(4.3). Analogously, we may verify that 
(C, R;))=0,for Bj; >0, Any solution X* of the problem (4.1)-(4.3) can be re- 
presented as 


X= J aX, + DF BAR: (4.10) 
Az=1 A=) 
where a, >0, A=1, 2,..., m, By >0, A=1,2,..., my Sia, = 1.Here 
a=i 
(C,X*)=(C,X,), Nee, By oe atts (4,11) 
(C, R;,.) =, A= 1, 2, eee, n\. (4.12) 


Let us isolate all the support solutions of the problem X;,4=1, 2,..., a, 
and all the vectors R;, A=1, 2, ...,a,, satisfying (4.12). Consider the poly- 


4! 
hedral set M* spanned by the points X, and the vectors R;,, i.e., the set 


of points X of the form 


nm, fz 
X= 2 a, X, + p2 Bi Ris (4. 13) 
a; 20, A= 1, 2, eee n,, B,, = 9, A= 1, 2, coey Ay, 


fy 


It follows from (4,10)-(4.12) that any solution x* of the problem (4.1)-(4.3) 
belongs to M*, Since the support programs X; and the vectors R, satisfy 
conditions (4.11) and (4.12), respectively, any point in M* solves the prob- 
lem in question. Thus, all the solutions of the problem (4.1)=(4.3) con- 
stitute a polyhedral set M* comprising points written in the form (4.13). We 
have thus proved the following proposition: 
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Theorem 4.3. All the solutions of the linear-programming problem 
constitute a polyhedral set spanned by the support solutions of the problem 
and those direction vectors R,; of the unbounded edges of M which satisfy 
the equality (4.12). 


From Theorem 4.3 it follows, in particular, that the linear-programming 
problem is either uniquely solvable or has an infinite number of solutions. 
In the latter case, however, all the solutions can be expressed in terms of 
a finite number of vectors X; and Rj. 

4-5, As we have indicated in Chapter 1, the linear-programming prob- 
lem may be unsolvable because the restraint system is inconsistent (the 
set of feasible programs is empty), or the linear form of the problem is not 
bounded (from above or below, depending on the statement of the problem) 
on the set of feasible programs. The following theorem shows that there 
are no other reasons which may render the linear-programming problem 
unsolvable. 


Theorem 4.4. (Solvability theorem of linear programming.) If the set of 
feasible programs of a linear-programming problem is nonempty and the 
linear form of the broblem is bounded from above on this set (the case of 
maximization), the problem in question is solvable, t.e., has at least one 
solution. 


Proof, Since the set of feasible programs is not empty, it is a polyhedral 
set and as such comprises points X of the form 


ky hk 
X= Yax,+> B, Ri, (4.14) 
= =1 
ky 
where all a,;>0,8;>0, >) a;=1, X,, i=1, 2,..., &, are some feasible programs 
'=1 
of the problem; R,, i=1, 2, ..., & is some system of vectors. 


Observe that we apply here the general representation theorem of poly- 
hedral sets (Theorem 2.6), which does not presuppose equality of rank of 
the system of restraints defining this set with the dimensionality a of the 
space of points X. The linear form of the problem, by assumption, is 
bounded from above. Therefore, for any vector R;,, 


(C, R;) <0 (4.15) 


(see (4.6)). 
Let X be an arbitrary program of this problem. From (4.14), this point 
can be written as 


X= X'+X’, 
Ry k 
where X’= )a,X; X"= > B,R; Applying (4.15) and noting that f; are nonne- 
(=1 f=1 
gative, we obtain 
(C, X)=(C, X)+(C, XH) S(C, X’). (4.16) 
We choose X; from the condition 
(C, X)= max (C, Xj). 
1slachk, 
In this case, applying the properties of the coefficients a, we have 


(C, X") a (C, Xy). (4.17) 
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Comparing inequalities (4.16) and (4.17), we obtain 
(C, X) <= (C, Xj") 


for any feasible program X of the problem in question. 

This inequality indicates that the program X; solves the problem. This 
completes the proof. 

Generally, Theorem 4.4 applies only to linear-programming problems. 
If we consider a problem involving maximization of a linear function de- 
fined on some convex closed set D, which is not a polyhedral set, Theorem 
4.4 does not generally apply. 

For example, maximize the form 


(4.18) 
subject to the conditions 


x,%,<=—1, x, >0. (4.19) 


The restraints (4.19) define in the (x,, x,)-plane a convex closed domain D 
bounded by the branch of the hyperbola x,x,=—1 lying inthe fourth quadrant. 
It is easily observed (Figure 2.3) that 


sup x, =0. 
(x1. %3)€D 


Here x,<0,and the point (+. —s) €D for any e>0. On the other hand, no 


point (x,,0)€D exists. The upper bound of the function (4.18) defined on D 
is therefore not attained on any point in D. 


£3, * CONS 
FIGURE 2,3 


Thus, all the requirements of Theorem 4.4 are satisfied in this problem. 
Nevertheless, the problem has no solution. Examples of certain discon- 
tinuous nonlinear functions defined on convex polyhedral sets can also be 
given for which Theorem 4,4 does not hold. 

We thus see that the assumptions of linearity of the function to be opti- 
mized, and the polyhedrality of its domain of definition required in Theorem 
4.4, are most significant, 

4-6. Until now we have considered the linear-programming problem 
written in arbitrary form. In describing methods of linear programming, 
we find it more convenient to deal with the canonical form of the problem 
(see Chapter 1, §5). We now proceed with the discussion of the general 
linear-programming problem given in canonical form. 

Maximize the linear form 


Dy cy%, (4.20) 
subject to the conditions 


n 
Pa 82) = bi i= 1, 2, coos MM, (4.21) 
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x20, fel, oa. (4,22) 


mon 
We remind the reader that the vectors Ay (Gy pr yyy cee, qs?) Jl, 2, 009M, 
and the vector B=(b6,, 6,,..., 6,)'are generally called the restraint vectors 


and the constraint vector, respectively. Equalities (4.21) are sometimes 
written more conveniently in vector form: 


Any B (4.21') 


Problem (4.20)-(4.22) is a particular case of the problem (4.1)—(4.3) for 
f=m, s=nfn, 


(0,0,...,0,—1,0, ..., 0), 0 esate eee / 2 
D;= 
(enya Gi_n,s) ep Ole als i=n+l, n+2, eos 
wee, AM, 
: 0, b= 1 2 eae Mi 
; bi_» é=n+1,a4+2, ..., a+m. 
Since the determinant of the vectors D,, D,, ..., D,does not vanish (it is 


equal to (—1)"), the restraint system (4.21), (4.22) is always of rank a. We 
shall assume that all the equalities of system (4.21) are linearly indepen- 
dent. This assumption does not limit the scope of our discussion, since 
otherwise some of the equations (4.21) could be omitted without modifying 
the polyhedral set of restraints of the problem. 

Let us find a support program for the problem (4.20)—(4. 22). 

Let X=(x,, x,,...,*%,) be a feasible program of the problem (4.20)—(4,. 22). 
According to the definition of a support program, there exist a linearly in- 
dependent vectors among the D, satisfying 


(D;, X) = dj. (4.23) 


The vectors D,,,,/=1,2,...,m are linearly independent and satisfy (4.23). 
Therefore, if (and only if) X is a support program, there exist among 
the vectors D;,, 1\<i«<n, n—m vectors D,, 1\<\<aa—m, which satisfy (4.23) and 

constitute, together with the vectors D;,,, |\i<m, a linearly independent 
system. This indicates that the determinant of the matrix whose rows are 
the vectors 

D, 


=— eh, lariqa—m, Disa = (Gis Bigy eos Bin), 


li<i<m, 


does not vanish. Expanding this determinant in the elements of the first 
n—mrows, we obtain the condition 


Ans Ais +++ Afg)l #9. (4,24) 
Here (A;, Aj,, ..-, Aj,) denotes the matrix consisting of the restraint vectors 
Aj,. The set of subscripts /,={/,, j,,---./a}is obtained from the complete set 
1, 2,...,2by neglecting the subscripts4, 1<\<n—m. Since x, =0, |<\<a—m, 


then, for x,>0,/€ ly. 

Hence, condition (4.24) can be expressed as follows: 

There exists a linearly independent system of m restraint vectors A, 
JEL=Uys Jay +++) Jn), Containing all those 4, for which x,>0. 

This requirement is, obviously, equivalent to the assumption of linear 
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independence of the system of vectors A, corresponding to the positive com - 
ponents of the program X. 

Thus, a support program for a canonical linear-programming problem 
can be defined as follows: 

A program X=(x,,x,, ..-,%,) of the problem (4.20)--(4.22) is said to be a 
Support program ifthe restraint vectors corresponding to its positive 
components are linearly independent. 

A system of m linearly independent restraint vectors comprising all the 
A, for which x,>0, is generally called the basis of the support program 


X=(x,, X%,,...,%,). The components of a support program connected with the 
basis vectors are sometimes called the basis components of this 
program. 


When solving linear-programming problems we have to deal not only with 
the vertices of the corresponding polyhedral sets, but also with the edges 
of these sets. The properties of edges of polyhedral sets were discussed 
in 1-4 (see Theorem 1.5) with reference to a linear-programming problem 
written in arbitrary form. We shall now try to apply these properties to 
the problem (4.20)-(4.22). Since the rank of the restraint system (4.21), 
(4.22) is 2, any edge of the polyhedral set of the problem (4.20)-(4.22) is 
either a ray or a segment, We know that the end points of an edge of a 
polyhedral set are vertices of this set (support programs of the correspond- 
ing linear-programming problem). 

The equation of any edge [I of the polyhedral set of restraints of the 
problem (4.20)—(4.22) can be written in the form 


X=X+Ae,, (4,25) 


where X is a support program of the problem, and e, is the direction vec- 
tor of the edge. The parameter 4 ranges from 0 to A,< oo if [I is a segment 
and from 0 to oo if fis a ray. 

Let E, be the set of subscripts i for which the equality (4.23) is satis- 
fied. Obviously, 2+i€£&, for!l<is<m. Since X is a support program, the 
rank of the system of vectors D,;, i€Ey,, is 2. According to the definition of 
an edge of a polyhedral set, the nonzero vector e,(the direction vector of 
the edge I) satisfies the following system of homogeneous equations: 


(Dp e,)=0, i=i,,4,, See eer a+i,n+2, woe, AM, 
leah an, LEE,, A=1, 2, eeeys n—m—y}. 


The vectors D,,, ..- Digegey Dagy ++) Day, are linearly independent. 
Let J,={h, Jay -++s Im» Ings} be the set of subscripts obtained from the set 
1, 2,..., awhere the subscripts i, 1<—A<n—m—1 are neglected. Remember- 
ing the specific features of the vectors D,; fori=1, 2,...,2, we formulate 
the preceding conditions to be satisfied by the vector e, as follows: 


(a) the vector e,=(e,,, @4,..+, é,,) Satisfies the equalities 


fn 
Pa, 2s =O =) Pa ee 
(b) for /€/,, &,=0; 
(c) among the restraint vectors A,, JEl, there exist m linearly independent 
vectors, 
Since the subscripts 4,€£y, \=1, 2,..., n—m—1, we have Iycl,. 
The basis of the support program X (the end point of the edge I) is thus 
obtained from the system of vectors A, j€/, by eliminating one of the vec- 
tors of tuis system. Assume that the vector 4A,,,, is to be eliminated. 
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In this case /,={j,,/,,..., jm}. If the basis of the support program X is given, 
the edge [ issuing from the vertex X is uniquely determined by the re- 
straint vector A;,,. Let us expand the vector A;,,, in terms of basis vec- 
tors of the program X: 


Aint, = = Aj_*. 
The coefficient x" can be used to establish whether the edge I is bounded. 
From the conditions determining the vector e, we have 


= (T) = 
Coja = Sosy r8 ? a=1,2,...,m. 


Substituting the above in (4,25) and taking into account that e,,=0 for /¢/, 
and x,=0 for j¢/,, we obtain 


x, — Ax?) Jojo Cel 2h cicy iM, 
“| A. J=Jnev (4,26) 
0, J€/,. 
Here X(A)=(x, (A), ...,%,(4)) is an arbitrary point of edge I’. Since the vector 


X (4) satisfies the restraints (4.21) for any A, this vector is a feasible pro- 
gram of the problem if and only if 


Xp (A) =X), — Axl) & 0, a=Il, 2, eoey m. 


If «<0 fora=1, 2,...,m, these inequalities are satisfied for any 4>0 and, 
consequently, IT is an unbounded edge (a ray issuing from the vertex X). If, 
however, x‘ >0 for some I, the edge a’is a segment, since 4 cannot ex- 
ceed the value A,, where 


The program X(i\,) gives the second end point of the edge [. As we have 
shown previously, X(4,) is a support program of the problem. It follows 
from (4.26) that the basis of the program X(A,) is formed from the basis of 
the program X by replacing one of the vectors A, j€/y, by the vector Ay,,,,:. 
The vertices of a polyhedral set which are end points of the same edge are, 
naturally, called adjoining vertices. The corresponding support 
programs are, therefore, sometimes called adjoining support programs. 
One of the most effective techniques of linear programming, the simplex 
method, deals with evaluation of the adjoining support programs of the 
problem, 

4-7. When describing various techniques of linear programming, it is 
often convenient to isolate the class of problems comprising the so-called 
nondegenerate linear-programming problems. We now give the relevant 
definitions. 

A support program of a linear-programming problem (4.1)-(4.3) is said 
to be nondegenerate if the number of restraints in the system (4.2), 
(4.3), reduced by this program to equalities, is nz. All these relationships 
should obviously be linearly independent. If the support program does not 
satisfy the requirement of nondegeneracy, i.e., if it reduces to equalities 
more than a restraints from (4,2), (4.3), itis calleda degenerate 
program. | 

A linear-programming problem is called nondegenerate, if all its 
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support programs are nondegenerate. A problem with at least one dege- 
nerate support program is calleda degenerate problem. 

These definitions are obviously meaningful only if the problem in ques- 
tion has support programs, i.e., if the restraint system (4.2), (4.3) is of 
rank a. 

If the support program X of problem (4.1)-(4.3) is nondegenerate, we 
may easily determine the number of edges issuing from the vertex X of the 
polyhedral restraint set. 

Assume that 


(D; X)=4; 
fori=i,, i, ..., é,-4s+1,s4+2, ..., s+ti<cii<s. Let the set of these subscripts 
be £,. By assumption, 
(D;, X)<d;, if iGLy (4.27) 


(X is a nondegenerate program), Lete=(e,,,..., 
whose components are solutions of the equations 


0, i€Ey, i#by, 
—1l, f=. 


é.,) |1<a<xn—t, be a vector 


(Dj, =) (4. 28) 


Consider the set [of points of the polyhedral set M written in the form 


X+Ae, AS. 


From equations (4.28) and inequalities (4.27) it follows that [, contains a 
certain segment, i,e., it is a one-dimensional set. Further, applying the 
first n—1 equations of system (4.28) we conclude that Il, is an edge of M with 
eas its direction vector. 

On the other hand, the direction vector e of any edge I issuing from the 
vertex X must satisfy the equations 


(D,, e)=0, i€ Ey, iti, 


for some a, |<a<xa—t, The vector e should therefore be parallel to one 
of the vectors e“ which, consequently, can be taken as the direction vector 
of the edge [. The edge [ thus coincides with one of the edges [Y,. 

Thus, precisely na—t edges of the polyhedral set Missue from each ver- 
tex X, which is a nondegenerate program ofthe problem (4.1)-(4.3). The 
direction vector of any such edge can be determined from equations (4,28) 
for some a, |xa<an—t, If the support program X is degenerate, the num- 
ber of edges of X issuing from the vertex M™ need not equal a—t. This 
number may be either smaller or greater than a—?t (see Exercise 12), 

Geometrically, nondegeneracy of a support program xX means that 
exactly 2 boundary hyperplanes of the polyhedral set M pass through the 
vertex X, If the problem (4.1)-(4.3) is degenerate an analogous property is 
characteristic of all the faces of M(and not only of the zero-dimensional 
ones). 


Theorem 4.5. If the problem (4.1)-(4.3) is nondegenerate, then through 
any q-dimenstonal face of the polyhedral set M pass exactly n—q boundary 
hyperplanes of M (all these are obviously linearly independent) . 

Proof. Let M,be any g-dimensional face of M. The face M,is a convex 
polyhedral set whose defining system of restraints is obtained from (4.2), 
(4.3), with some of the inequalities in (4.2) replaced by equalities. All the 
inequalities appearing in the restraint system of M,can be considered as 


68 


imposing nonrigid constraints on M, Therefore, according to Theorem 
1,2, the rank of the equality restraints in the defining system of M, is 2a—g. 

Let X, be an arbitrary vertex of the polyhedral set M, The existence 
of this vertex follows from the fact that the rank of the restraint system de- 
fining M, is a(equal to the rank of the restraint system of the polyhedral 
set M). It is easy to verify that X,is a vertex of the polyhedral set M (see 
point 2 in the proof of Theorem 2.2). By assumption X, is a nondegenerate 
program. Accordingly, all restraints (4.2), (4.3) reduced by this program 
to equalities should be linearly independent. In particular this also applies 
to the equality restraints of the polyhedral set M, Since the rank of these 
restraints is nza—q, their number also equals na—g. This completes the 
proof. 

Consider now a canonical linear-programming problem (4.20)—(4.22). 
Let X be a support program of this problem. If the restraint vectors A,, 
J€ly, constitute the basis of program X, then x,=0for /@ly. This indicates 
that the program X reduces a linearly independent restraints of the de- 
fining system of the polyhedral set Mto equalities 


a, ,X,=b,, i=1, 2, oo, M, 
+, =0, dtly={h,; Jy = erie 


The requirement of nondegeneracy of the program X reduces all the other 
restraints of the problem to strict inequalities. Hence, a program X is 
nondegenerate if and only if 


x,>0 for /g/y. 


Thus, the nondegeneracy of a support program of a canonical linear- 
programming problem can be defined as follows: 

A support program X of the problem (4,20)-(4.22) is said to be nonde- 
generate if all its components corresponding to the basis vectors (basis 
components) are positive (x,>0Ofor jeé/y). Observe that the basis ofa 
nondegenerate support program is determined uniquely as the system of 
vectors corresponding to the positive components of the program. A dege- 
nerate support program may have several bases. 


§ 5. The geometry of the linear-programming problem 


5-1. As we saw in Chapter 1, geometrical considerations are very use- 
ful in the analysis of linear-programming problems. In the following the 
reader will see that geometrical analogies make the techniques of linear 
programming much more lucid and constitute the heuristic basis of the 
various theorems in linear programming. A new approach to the solution 
of linear-programming problems often arises from elementary geometrical 
considerations. In this section we shall deal with two geometrical inter- 
pretations of the general linear-programming problem with any number of 
restraints and unknowns. We suggest that the reader, before carrying on 
with this section, review Chapter 1, § 6 where the two interpretations are 
considered with reference to two- and three-dimensional cases. The cor- 
responding drawings (Figures 1.1-1.5) given there serve as good illustra- 
tions of the somewhat abstract discussion that follows. 
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5-2. We start with the first geometrical interpretation of the linear- 
programming problem. Consider the general linear-programming problem 
(4.1)-(4.3) written in arbitrary form. The restraints (4.2), (4.3) define a 
convex polyhedral set M in the n-dimensional space of points X=(x,, x,, ..., x,) 
(always assuming the restraints to be consistent). The polyhedral set M 
may be considered as the intersection of the half-spaces 


(D, X)<d;, i=1,2,....8 
and the hyperplanes 
(D;, X)=d;, i=s+1, s4+2, ..., s+. 


The boundary of Mconsists of parts of the boundary hyperplanes (D,;, X)=4d,, 
i=1,2,..., s+#. Observe that, in general, not all the boundary hyperplanes 
are part of the boundary of M. The hyperplanes defined by the inequality 
restraints (4.2) need not have common points with M. Thus the restraints 
(4.2) defining such hyperplanes could be omitted without affecting the poly- 
hedral set M; however, it is most complicated to find these restraints 
analytically. 

The dimensionality @ of the polyhedral set M is not greater than a—f, 
where t is the number of equality restraints (4.3) which are assumed to be 
linearly independent. If all the restraints (4.2) are nonrigid constraints of 
M, then g=n—t. Therefore, translating the origin of coordinates to some 
point of intersection of the hyperplanes 


(D, X)=d;, t=stt,..., sth, 
we may consider M in an (a—t)-dimensional subspace of the principal space. 
Analytically, this can be achieved by expressing any ¢ variables from equa- 
tions (4.3) in terms of the other n—t variables and subsequently eliminating 
the former from the inequalities (4.2), The space containing the set M thus 
becomes (2—1t)-dimensional. This technique was already employed in 
Chapter 1, §6. The linear form (4.1 ) of the problem defines, in the az- 
dimensional space, a system of parallel hyperplanes 


(C, X)=ih, —oo< A < oo, 


Each of these hyperplanes will be called the linear-form hyper- 
plane of the problem. The linear-form coefficients ce, j=l, 2,..., acon- 
stitute a vector C=(¢,,¢,,...,¢,) orthogonal to the family of hyperplanes. The 
vector C points in the direction of increasing linear form. With fixed Aj, 
the linear-form hyperplane generates two half-spaces. One of these half- 
Spaces containing the point X+C (the point X belongs to the hyperplane) will 
be called the upper half-space, while the other half-space will be 
called the lower half-space. The equation of the upper half-space 
has the form 

(C, X) Si, 


and the lower half-space is defined by the equation 
(C, X)<A, 


For A=A,, let the linear-form hyperplane intersect the polyhedral set 
of restraints M. The linear form has the same value at all the points of 
intersection. Translating this hyperplane parallel to itself along the vector 
C (in the direction of increasing linear form (4.1)), it may happen that fur- 
ther translation will cause the hyperplane and the set M to be disjoint. 
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Let this limiting position of the hyperplane correspond to 71=A,, i.e. , 
the corresponding hyperplane has the equation 


(C, X)=A,. (5.1) 


In this case the polyhedral restraint set of Mis located in the lower half- 
space of hyperplane (5.1). At each of the points belonging to both M and 
the hyperplane (5.1) (the set of these points is a priori nonempty) the linear 
form attains the extremum 4A,. The intersection of Mand the hyperplane 
(5.1) defines a polyhedral set M*of solutions of the problem in question. 

If the problem has a unique solution, M* comprises only one single 
point, the vertex of M. In the general case, M* is some convex polyhedral 
e*dimensional set, so that 


0=<e*<2@. 
Observe that e*=e only if M belongs to one of the hyperplanes of the family 
(4.29). 

Until now we have assumed the existence of the limiting position of the 
linear-form hyperplane, as defined by the equality (5.1). This assumption 
is obviously justified only for a bounded M(if Mis a convex polyhedron). 

If, however, M is an unbounded convex polyhedral set, then even if the 
linear-form hyperplane is displaced an arbitrarily large distance in the 
direction specified by the vector C, it may still intersect M. This means 
that the linear form of the problem is unbounded on the set M, i.e., the 
given problem is unsolvable, It must be kept in mind that the fact that Mis 
not bounded is not a sufficient condition for the unsolvability of a problem. 
For some values of the vector C the problem with an unbounded polyhedral 
restraint set is solvable (the linear-form hyperplane has a finite limit 
position), and for others it is unsolvable (there exists no finite limit posi- 
tion of the linear-form hyperplane). 

Sometimes an erroneous statement of the problem results in inconsistent 
conditions (4.2 ), (4.3). Geometrically this corresponds to the case when 
the domain of definition of the linear form degenerates to an empty set. 

5-3, The first geometrical interpretation of the linear-programming 
problem applies equally well to any representation of the problem. The 
second geometrical interpretation, which we give below, is suitable only for 
the canonical form of the problem. 

Consider a general linear-programming problem in canonical form 
(problem (4.20)—(4.22)). 

We introduce new variables g,, 4,,...,4,, U,,, So that 


n 
U,= 2) Aj jXp i=1, 2, ..., mM; 
fe (5.2) 


ft 
Uns = Py C X70 


Relationships (5.2) define a transformation of the n-dimensional space of 


points X=(x,,x,,..., *,) to the (m+ 1)-dimensional space of points U=(u,, 
U,, ..., Uay,). Let A; denote the (m+1)-dimensional column vector with com - 
ponents 4), Gy, ..+) gy Le, 

A, = (24, asp eee, any c,)". 


The vector A, whose first m components coincide with the components of 
the restraint vector A, was called in Chapter 1 the augmented re- 
straint vector. 
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Relationships (5.2) can be rewritten in vector form 


gs J 
U= 9 Ap, (5.3) 
p> ad) 
By definition, the set of points U representable in the form (5.3) with 
x,20, /=1,2,..., a, is a convex polyhedral cone (see the second definition of 


a polyhedral cone in 3-5). Denote this cone by K. The cone K is spanned 
by the augmented restraint vectors 4,; the apex of K is located at the origin. 
It follows from (5.3) that K is the image of the positive orthant of the space 
of points X in the (m+ 1)-dimensional space of points U. 

Now,, let a point X satisfy the equalities (4.21). Then, according to (5.3), 
the image of this point in the (m+1)-dimensional space in question is the 
point 

Uy=(,, Oy, «2+ Og, L(X)), 


where L(n= 3 ¢,x; On the other hand, for any 4, among the solutions of 
et R 

the system (4.21), there exists a solution X so that A=L(X)= p2 cx, (We 
=1 


assume here that the vectors C and (a,,, a,,,...,4@;,), #=1, 2,...,mare linearly 
independent, and m<n.) 
Hence, (5.3) transforms all the solutions of the system (4.21) to the line 
Q whose equation is = 
U=B+)egs 4, —0O <A < 00, 


where B=(b,, ),, ...,6,,9), &ms,=(0,0, ...,0, 1). The line Q passes through the 
point B and is parallel to the Ou,,,-axis. 

Since the feasible programs of the problem (4.20)-(4.22) must satisfy 
the equalities (4.21) as well as the inequalities (4.22), the transformation 
(5.3) maps the polyhedral restraint set M onto the intersection of the cone 
K and the line Q. 

Assume that Mis nonempty. The cone K and the line Q then have com- 
mon points. Let the intersection of the line Q and the cone K be denoted 
by Q,. The intersection of convex sets is convex. The dimensionality of 
the set Q, is not greater than unity. Hence, Q, is either a line, ora half- 
line (ray), or a segment, which may degenerate to a point. 

To each point U=(6,, b,,...,5,,4) € Q, there corresponds a set of points 
X€M so that (C,X)=A’. The linear-programming problem entails finding a 
point X*€M maximizing (C, X). In the (m+1)-dimensional space, solving the 
problem is equivalent to finding a point U*€Q, with maximum possible 
(m+1}-th coordinate A* 

Two cases must be distinguished here: 

1. There exists 4 so that for any point (6, b,,...,b,A)EQ 

a 

2. The set Q, contains points with arbitrarily large values of the (m+ 1)-th 
coordinate, 

In the first case Q, is either a segment or a ray with direction vector 
-@,,,(a ray pointing along the negative Ou,,,-axis). The problem in ques- 
tion is solvable and its optimal polyhedral set M* (the set of all the solu- 
tions) corresponds to the upper (in the sense of the Ou,,,-axis) end point 
of Q, i.e., the uppermost point of intersection of the cone K and the line Q. 

If the set Q,is a line or a ray with e,,, as the direction vector, we have 
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the second case. This obviously indicates that the linear form of the prob- 
lem is unbounded on the set of feasible programs. 

It may occur that the line Q lies outside the cone K so that the intersec- 
tion of Q and K is the empty set. In this case the problem has no feasible 
programs, i.e., the restraint system (4.21), (4.22) is inconsistent. 

Thus, the solution of problem (4.20)-(4.22) in terms of the second geo- 
metrical interpretation entails finding the uppermost point of intersection of 
the line Q and the cone K. When Qand K have no common points, the 
problem is unsolvable since the restraints (4.21), (4.22) are inconsistent. 
If, however, the intersection of Q and K is a nonempty set having no upper- 
most point, the problem is unsolvable because the linear form (4.20) is 
unbounded on the polyhedral restraint set M. 

In our geometrical interpretation of linear-programming problems, we 
have limited the discussion to maximization problems. All the foregoing 
applies, with obvious modifications, also to minimization problems. The 
reader will easily perform these modifications, if necessary. 


EXERCISES TO CHAPTER 2 


1, M,is an a-dimensional cube, i.e. , a convex polyhedron, defined in the space of points X=(x,, 
X, «++, %,) by the restraints 


Oaxxa, a>O0, jm 1, 2, ..., 8% 


Determine the number of @-dimensional faces of M,, O<gacn. 
2. M, is an n-dimensional simplex, i,e., a convex polyhedron, defined in the space of points X=(x,, 
Xs, --+, X,4,) by the restraints 


x; => 0, j=1, 2, .oey AGT. 


Determine the number of g-dimensional spaces of M,, where Om gan. 
3. Solve the system of linear inequalities 


x,—2x,+ x,<1, 
2x, + %3— x= 2, 
3x, + 2x,—2x, <3, 
X%,— %— x,<1, 
2x, +3x,+ 2x, < 2, 
giving its general solution, 
Hint. Apply the representation theorem of polyhedral sets. 
4, Let the faces [, and I of polyhedral sets M, and M _ be corresponding in the sense of 2-3, Prove 
that their dimensionalities @(T,) and @(I) are related by 


e(Py=e(l.) ta—r, 

where a is the dimensionality of the space and rf is the rank of the restraint system defining the polyhedral 
set M. 

5, Prove that in the n-dimensional vector space there exist n+-1 vectors so that all their nonnegative 
linear combinations span the given space, Show that any / vectors with /< do not have this property, 

6. Prove that a convex polyhedron may be defined as a closed bounded convex set containing a finite 
number of extremum points, 

7, Show that any convex polyhedral set can be represented as a sum of some convex polyhedron and 
some convex polyhedral cone with apex at the origin. 

8. Show that a convex polyhedral set with vertices isa convex hull* of its vertices and unbounded edges, 

9, Prove that a convex polyhedral set M is a convex hull of its a—r- and a—r +1 -dimensional faces. 
Here n is the dimensionality of the space of points X and f is the rank of the restraint system (1.1), (1.2) 


* (For the definition of a convex hull, see p.494. ] 
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defining M. Show that in this representation only such (a—r+1)-dimensional faces need be retained which 
contain one n—r-dimensional face, 

10, Prove that a convex polyhedral cone, which is not an intersection of several hyperplanes, coincides 
with the convex hull of its a—r+1-dimensional faces, where a and r are defined as in the preceding 
exercise, 

11, Give an example of a continuous function defined on a convex polyhedral set to which Theorem 4.4 
does not apply. 

12. If the support problem X of (4.1)-(4.3) is degenerate, the number of edges of the polyhedral set M 
issuing from the vertex X may be either less, or more than a—f, Construct suitable examples. 

13, Show that an arbitrary linear transformation 


Y=AX, 


where A is a n, Xn, matrix, transforms any convex polyhedral set of the n,-dimensional space of points X 
into a convex polyhedral set of the a,-dimensional space of points Y, 
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Chapter 3 
DUALITY 


This chapter deals with the principles of one of the basic concepts of 
linear programming — duality. 

Associated with every linear-programming problem is a corresponding 
(linear-programming) problem, generally called the dual or the con- 
jugate problem. Duality establishes a close relationship between the two 
problems constituting a unique dual pair. Simultaneous analysis of dual 
pairs proves to be useful both in developing numerical methods of linear 
programming and in qualitative investigations in linear programming and 
related mathematical subjects. 

We present the theory in the following sequence. In §1 a general state- 
ment of the problem is given and some elementary dual relationships are 
established. In § 2 certain properties of convex polyhedral cones, essential 
in the proof of duality theorems, are derived. In proving the theorems 
presented in §3 for problems with homogeneous restraints we use the lucid 
geometrical interpretation as opposed to the conventional technique (see, 
e.g., /25/). In §4 duality theorems are extended to linear-programming 
problems with mixed restraints. In §5 we deal with the relationship be- 
tween the solutions of the dual problem and the so-called decision mul- 
tipliers. Optimality criteria of programs are formulated in terms of 
decision multipliers. In this section we also give two significant interpre- 
tations of decision multipliers. According tothe first interpretation deci- 
sion multipliers can be considered as analogous to Lagrange multipliers, 
and according to the second, decision multipliers are used to assess the 
influence of the right-hand sides of the restraints on the maximum attain- 
able value of the associated linear form. 

In the last section (§ 6) duality theorems are used to establish certain 
sufficiency conditions for the unique solvability of the linear-programming 
problem and to prove certain propositions of the theory of linear inequalities. 


§ 1. Statement of the problem 


1-1. Consider a general linear-programming problem in canonical form. 
Maximize the linear form 
n 


L(X) = Bey; (1.1) 


=! 
subject to the conditions 


241; %;= by, §=1,2, 06. mM; (1.2) 
j= 


75 


x, = 0, J=1,2,..6,8. (1.3) 


In conjunction with problem (1.1)—(1.3) we introduce another linear-pro- 
gramming problem. 
Minimize the linear form 


L(Y) =X biyr (1.4) 
=1 
subject to the conditions 

24,91 > Cy, J=1,2,.00,2 (1.5) 


Problem (1.4)~(1.5) is generally called dual or conjugate with 
respect to problem (1.1)—(1.3). Problem (1.1 )-(1.3) is termed the primal 
problem. 

We rewrite the restraints of problems (1.1)—(1.3) and (1.4)=—(1.5) in 


matrix form. Let 
A=||a;,|] =(A,, Ag, +--+» Ag) 


be the restraint matrix of problem (1.1)-—(1.3) comprising the restraint vec- 
tors A, of this problem. Let, as before, 


Babi byy eee y Og), CHE(Cys Cyr ee ey Cy)s 


With these notations, the above problems take the following form: 
Primal problem. Determine an a-dimensional vector X=(x,,x,,...,%,)7 
maximizing 


L(X)=(C, X) (1.1') 

subject to the conditions 
AX = B, (1.2") 
X>0. (1.3') 


Dual problem. Determine an m-dimensional vector Y=(y,, y,...,y,)7 
minimizing 
L(Y) =(B, Y) (1.4!) 
subject to the conditions 
ATY>C. (1.5') 


The dual problem is, thus, obtained from the primal problem when: 
(a) the vectors B and C are interchanged; 

(b) the matrix A is transposed; 

(c) the equalities of the restraints (1.2') are replaced by inequalities; 
(d) the restraints (1.3') are eliminated; 

(e) maximization is replaced by minimization. 


We will now illustrate the above by a numerical example. 
Consider a linear- programming problem which entails maximization of the linear ‘stag 


L (X) =x, +3x_ + 2x,—3x,—x, (1.6) 


subject to the conditions 


2x, + 2g tXyt Xet2x5+ x=, \ ee 
4x, +3x,—x,—2x,— x, +2,=1. (1.7) 
x, 20, j=1,2,...,6. (1.8) 


76 


Here 


22 1 1 21), oe 
Am(43 1 —2-p a) BHC N% CHUL, 3, 2,0,—-3,—1). 


According to the general rules, the dual problem of problem (1.6)~(1.8) is formulated as follows. 
Minimize the linear form 


LY=n4+H (1.9) 


subject to the conditions 
2y,+4y,> 1, 
24, + 3y, > 3, 
W— YW, 2, 
yy, 0. (1.10) 
24,—- > —3 
9, + 29, —1. 


1-2, We now give the geometrical interpretation of the dual problem. We 
first recall the geometrical meaning of the problem (1.1)-(1.3) according 
to the second geometrical interpretation (see Chapter 2, 5-3). The re- 
straints of problem (1.1)-(1.3) define a linear transformation of the a-di- 
mensional space of points X=(x,, x,,..., x,) onto the (m+ 1)-dimensional space 
of points U= (u,, u,,..., U_4,)S0 that 
n 


A= p> a, Xj i=, 2, eeovy mM, 


n 
ey, C,Xp 
m+ p> 1s 
The image of the positive orthant of the X-space is a convex polyhedral 
cone K, with apex at the origin, generated by the augmented restraint vec- 


tors A, A,,...,A,. We remind the reader that 


A ;= (8) ayy eet, ayy eT. 
The image of the set consisting of all the solutions of (1.2) is a line Q 


passing through the point B=(b,, b,,..., 5,, 0 parallel to the Ou,,,-axis. The 
equation of the line Q is 


U=B+hie,.,,, —oo<i<oo, (1.11) 
where 
One, = (0, 0, eee 0, 1). 
od 


mM 
Problem (1.1)-(1.3) entails finding the "upper" points of intersection of the 
line Q and the cone K. 
Consider the set of all the hyperplanes in the (m+ 1)-dimensional space 
of points U passing through the origin. The equation of any of these hyper- 
planes is 


2a batt, = 0. (1.12) 


The hyperplane (1.12) and its direction vectors A=(A,, Ay... Mugs Agas) 
specified to an arbitrary (nonzero) factor, are mutually orthogonal. For 
our purposes it is convenient to impose on the direction vector A the addi- 
tional condition 

Aenea. (1.13) 


This condition eliminates hyperplanes parallel to the Ou,,,-axis. From 
condition (1.13), there is a one-to-one correspondence between the hyper- 
planes passing through the origin and not containing the Ou,,,,-axis and their 
direction vectors. 
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Let Y«(y,, y,,--+. ¥,)be a feasible program of problem (1.4)-(1.5), i.e., 
a vector satisfying conditions (1.5). Consider the hyperplane Il, defined by 
the equation 


m 


m Yi 4ji— Ung =O. (1.14) 
We shall show that the cone K is located in one of the half-spaces generated 
by Il,. 

According to (1.5), which the vector Y satisfies, insertion of any of the 
A, (V=1, 2,..., a)in the left-hand side of (1.14) gives a nonnegative number. 
Therefore, the position of the cone K generated by the vectors A,, /=1, 
2,..., a, relative to the hyperplane Il,is indicated by the vector —e,,,=(0, 
0,..., 0, —1), corresponding to the negative direction of the Ou,,,-axis. The 
cone XK is thus below the hyperplane II, (in the sense of the Ou,,,-axis). 

Consider now an arbitrary hyperplane passing through the origin and not 
containing the Ou,,,-axis. Let its equation be given by (1.14). 

If the cone K is below the hyperplane (1.14), the vector Y=(y,, .:., Yn) 
defining this hyperplane satisfies the conditions (1.15) and, consequently, 
is a feasible program of the dual problem (1.4)-(1.5). To verify this pro- 
position, it is sufficient to insert in the left-hand side of (1.14) the coordi- 
nates of the augmented restraint vector 


A, (j= 1, 2,..., 0), 


which, by assumption, is above the hyperplane in question. 

Thus, geometrically, the set of feasible programs of the dual problem 
coincides with the set of hyperplanes passing through the origin and situated 
above the cone K. Moreover, there is a one-to-one correspondence, de- 
fined by equation (1.14), between the programs /Y of the dual problem and 
the hyperplanes II, of the set. 

Let us determine the (m+1)-th coordinate, um,,, of the point of intersec- 
tion of the line Q and the hyperplane IIy. Applying (1.11) and (1.14), we 


obtain 
m 


Une b,y;= L(Y). (1.15) 
The relationship (1.15) shows that the value of the linear form of the dual 
problem of program Y equals the ''distance" of the point of intersection of 
the line Q and the hyperplane [l, from the hyperplane 4u,,,=0 (the word 
distance’ appears in quotation marks because uy, may be either positive 
or negative). 

We now give a geometrical interpretation of the dual problem. Geo- 
metrically, the dual problem (1.4)-(1.5) consists of finding a hyperplane 
passing through the origin and situated above the cone K which, moreover, 
intersects the line Q at the lowest point (relative to the Ou,,,-axis). 

1-3. For m=2the above geometrical considerations can be easily visua- 
lized, 

Consider problem (1.6)-(1.8) with two equality restraints (1.7). In this 
case the augmented restraint vectors have the form 


78 


The cone K is generated by the vectors A,, j=1, 2, 3, 4, 5, 6 and is located in 
the three-dimensional space of points U=(u,, u,,u,). The edges of the cone 
K are the rays 


U=A,\, ADO, j=l, 2, 3, 4, 5, 6. 
The equation of the line Q, according to (1.11), is 


“u,=1, 
ue=1, (1.16) 
u,=h, 


where —co <4 < oo. 

Let the cone K be intercepted by a plane passing through the point (1, 0, 
0) and perpendicular to the Ou,-axis. The equation of this plane is u, =1, 

The intersection of the plane w,=1 and the cone K is a polygon a,a,a,a,a,a,. 
The vertices of the polygon a,a,a,e,a,a, are the points of intersection of the plane 
u,=1 with the corresponding edges U=A,), A=0,0f the polyhedral cone K. 

The polygon a,a,a,a,a,a, is shown in Figure 3.1. In this figure the line Q 
contained in the uz,=1 plane is also shown, 

Each plane passing through the origin and not containing the Ou,-axis in- 
tersects the plane u,=1 along a line not parallel to Q, These lines are known 
as the traces of the corresponding planes. Figure 3.1 shows four such 
traces: 

SR; SR; S’R", S*R*, 


We observe that any line in Figure 3.1 is the trace of a plane corresponding 
to a feasible program of the dual problem (1.9)—(1.10) if and only if the poly- 
gon a,@,a,a,a,a, is below this line. In particular, the traces RS and R*S* cor- 


respond to feasible programs of the dual problem. The two other traces 


FIGURE 3.1 


do not specify feasible programs of the dual problem: R’S’ passes below 
the polygon a,a,a,a,a,a,, and R’S”"'cuts'' the polygon in two, The geometrical 
image of the solution of the primal problem is the point a, namely the upper 
point of intersection of the line Q and the polygon a,a,a,a,a,a,. 

The solution procedure for a dual problem consists of finding the trace 
lying above the polygon a,a,a,a,a,a, and intersecting the Q-axis at the lowest 
point. Geometrically, it is obvious that these requirements are fulfilled 
by R*S* whichis a supporting line of the polygon a,a,c,a,a,a, at the point a, 
Hence, it follows that the linear forms of problems (1.6)-—(1.8) and (1.9)—(1.10) 
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coincide with their respective optimal programs. The optimal value of 
either linear form is equal tothe length of the segment aa’(see Figure 3.1). 

In the following we shall see that the above properties of problems (1.6)— 
(1.8) and (1.9)-(1.10) are true for any linear-programming problem. 

To clarify the geometrical meaning of the problem (1.9)-(1.10) we could, 
naturally, use the cone K, and not necessarily the intersection of the cone 
with the plane u,=1 containing the Q-axis. The transition to the polygon 
a,0,a,a,a,a,was effected only to make the discussion less abstract. 

Observe that the plane u, =! could be replaced by any plane containing 
the line Q. 

1-4, In Chapter 1, § 7 we gave an economic interpretation of the linear- 
programming problem and only considered the class of problems with in- 
equality restraints and nonnegative variables. Problems of this class are 
called problems with homogeneous restraints. The economic inter- 
pretation of the dual problem (see 1-5) applies only to problems with homo- 
geneous restraints. Anarbitrary linear-programming problem with homo- 
geneous restraints is formulated as follows: 

Maximize the linear form 


Dy 6%; (1.17) 
jel 
subject to the conditions 
> @,,X, <b, i=l, 2, eeey mm; (1.18) 
i= 
x, = 0, J=1, 2, ceeg R. (1.19) 


Problem (1.17)-(1.19) is easily reduced to canonical form. To this end 
it suffices to introduce additional nonnegative variables x,,,, i=1,2,..., m, 
and to rewrite the restraints of the problem (1.17)=(1.19) in the equivalent 
form 


p> BX s+ Xq4 = Oi, i= 1, 2, eee, m; (1.18') 
xj20, s=l, 2,..., +m. (1.19') 


The equivalent problem has na+m nonnegative variables and m equality re- 
straints. 

Applying the general rules we formulate the dual problem of (1.17), 
(1.18') and (1.19'). 

The augmented restraint vectors of the primal problem are 


He asp eee, Cay c,), if Iayjacn, 
A:;= 


; (0, O,...,0, 1,0,...0)%, if rz+iejacat+am. 


The dual problem is therefore formulated as follows. 
Minimize the linear form 


m 
Dd Fim (1.20) 
sot 
subject to the conditions 
2 HH ep J=1,2, o005 M3 (1.21) 
=1 
y¥, =, é=1,2, ..., m. (1.22) 


It must be emphasized that the dual problem (1.20)-(1.22) is a problem 
with homogeneous restraints. 
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Introducing new parameters a,,=—4;, b,=— by ¢,=—c, we reduce the 
problem (1.20)—(1.22) to the maximization of the linear form 


py bY; (1.20') 
axl 
subject to the conditions 
2 HSE J=1,2, 0.5% (1,21') 
y, 20, t=1,2, ..., m. (1.22!) 


The ensuing problem is identical in form to the problem (1.17)-—(1.19). 
Hence, its dual problem involves the minimization of the linear form 


na 


3 *; (1.23) 

subject to the conditions “a 
3) aix,0,, t= 1,2,..., m (1,24) 
= x, 0, juli, 2,..., a. (1.25) 


If in problem (1.23)-(1.25) we revert to the original parameters a,,, 6,,¢, 
we obtain the primal problem (1.17)-(1.19) which is thus the dual of the 
problem (1.20)-—(1.22). Consequently, the problems (1.17)-—(1.19) and (1.20)-— 
(1.22) may, obviously, be considered as a dual, or conjugate pair. 
Each problem of this pair is thus dual with respect to the other. To single 
out one of the problems of a conjugate pair as the primal problem is thus, 
to a certain extent, arbitrary and purely formal. 

In § 3 we shall see that the problems (1.1)-(1.3) and (1.4)-(1.5) also con- 
stitute a dual pair. 

1-5. We now give an economic interpretation of the problem (1.20) 
(1.22), dual with respect to the problem (1.17)-—(1.19), First we recall the 
economic interpretation of the primal problem (1.17)-—(1.19) given in Chap- 
ter 1, 7-2. 

There are az modes for producing a certain homogeneous commodity, of 
which c, elements are produced by the j-th production mode in unit time. 
Operating by the /-th production mode for unit time involves the consump- 
tion of the i-th production factor (i=1, 2, ...,m)in the amount of Gy; 

Let the resources of the production factors be 6,,6,,...,5, units, respec- 
tively. Let x, be the time scheduled for the /-th mode of production. 

Problem (1.17)—(1.19) is the mathematical statement of the problem 
dealing with the determination of a scheduling program of the various pro- 
duction modes ensuring a maximum output of the homogeneous commodity 
with the given resources (8,, 5,, ..., 5,). 

Now, suppose that within the framework of the problem specified we 
have to evaluate each of the production factors. We shall consider here only 
an idealized self-contained model of production where the external relation- 
ships are strictly specified by the conditions of the problem. 

The problem restraints (limited resources and available production 
modes) enable us to evaluate each of the factors. It should be kept in mind 
that this evaluation is characteristic for the industry in question and as such 
is relative. Similar production factors are different in significance in dif- 
ferent industries and localities. In 5-5 we shall see that factor evaluation 
is a measure of usefulness of a particular factor for the given industry 
under strictly specified conditions. 
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Any change in production conditions, in particular a change in the re- 
sources of the various factors, makes it necessary to reassess these fac- 
tors. The evaluation of the production factors is relative because these 
factors are assessed in units of value of the output commodity. The value 
of the commodity is determined by conditions which are not inherent in the 
production process. 

Let the cost of a unit of the output commodity be taken as unity. The 
cost of a unit of the output commodity (the production cost) is introduced 
here as a basic concept proceeding from which we shall evaluate the various 
production factors. Let y, (i=1,2, ...,m) be the cost of the i-th production 
factor. 

We now study the j-th mode of production in terms of cost and profit. 

Producing under this mode for a unit time, we incur an overall cost 


m 
2,= Dis Iin 
é=1 


whereas the cost of the output commodity is c,, Given the correct values of 
the production factors, the overall expenditure cannot be less than the cost 
of the output commodity, since otherwise the goods would have been partly 
produced from "nothing''. Hence, for any j=1, 2, ....4 

2, C,. 
In other words, the cost vector Y=(y,, y,,..., y,) should satisfy conditions 
(1.21). 


Moreover, the costs y,, yo. ..-»¥, Should obviously be taken as nonnegative 
numbers. The vector Y=(y,, y,,..,y,), therefore, also satisfies condition 
(1.22). 


The cost vector of the production factors is thus a feasible program of 
the dual problem (1.20)—(1.22). 

However, conditions (1.21), (1.22) do not completely specify the cost vec- 
tor Y. When interpreting the problem (1.17)-—(1.19) economically, the para- 
meters a;, Should naturally be taken as nonnegative numbers and, for any ,, 
at least one of the a,, should not vanish. Any vector with sufficiently large 
components is, therefore, a feasible program of the dual problem (1.20)— 
(1.22). 

We must, therefore, find a condition which would eliminate unjustified 
exaggeration of the cost of the production factors. An obvious restriction 
of this kind is the following. The cost vector Y should be such that the 
overall value of the resources, 


m 
> 5: Yi, 
$= 


at the disposal of the factory be as small as possible. 

In what follows we shall see that if this requirement is not met with, the 
cost of the output commodity, for any scheduling program of the production 
modes, is invariably less than the value of all the resources. Conversely, 
if the foregoing condition is satisfied, there exist production programs en- 
suring equality of the cost of the output commodity to the resources avail- 
able. This point also justifies the introduction of the last restraint. 

The dual problem (1.20)—(1.22) is thus a mathematical statement of the 
problem of correct evaluation (costing) of all the production factors. The 
cost vector of the production factors solves the dual problem. In the fol- 
lowing we shall sometimes call the feasible programs of the dual problem 
tentative cost vectors of the production factors. 
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The dual pair (1.17)-(1.19) and (1.20)-(1.22) is conveniently depicted in 
the following tableau (Table 3.1). 


TABLE 3.1 
Productivity 
C, Ce Cy Cr 
x Gr | Ay Gyo o-- Gyy Qin | 0, 
2 | Ya | Se Gee oe Oey Gn | b| § 
GW ee nage we Sens de tee Saag es Og wa oe 2 
. 5 
|W | an Gis «+ Gry Gin | | 2 
'@) oeeeeee e e of 
Yn) Fm Fnsa--> Omj Amn| Om 


x; Xy eee x; eee Xn 


Production program 


1-6, To conclude this section we advance some simple, but nevertheless 
most useful, propositions relating to primal and dual programs. These 
propositions are repeatedly used in the following. 

Lemma 1.1. If X =(x,, x,,...,x,) and Y=(y,,y.,.-,y,) ave feasible programs 
of problems (1.1)-(1.3) and(1.4)-(1.5), respectively, then 


n ™% 
p2 ce, =D b, yy. (1.26) 
=i (=1 
Proof. By assumption, 
m 
D 2p Wi Cp j= 1, 2, oo ee M 
f=1 


Hence, 
n m m n 
pa eS > x, F %jx=Z “p> Ay, Xz 
j=l j=1 {m1 (=) 3 


The vector X is a feasible program of problem (1.1)-(1.3); therefore 


»» a,,*x,=b, for i=l, 2,...,m. 
=! 


Applying these equalities, we transform the right-hand side of the preceding 
relationships and obtain 


Rn m 
=1 emi 


This completes the proof. 

From Lemma 1,1 it follows, in particular, that if we take a feasible 
program of the dual problem (which is not its solution) as the cost vector 
we shall not be able to equate the cost of the output commodity and the re- 
sources available. 

Indeed, if Y is a nonoptimal production program of the dual problem, 
and Y* is the solution of this problem, then for any primal program X 


a m m™ 
>» CXS 2, byi< 2 b; Yj. 


Lemma 1.2. If for some programs X* =(x*,x*,..,x*) and Y* =(y*, y®,..., y%) 
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of problems (1.1)-(1.3) and (1.4)-(1.5), respectively, 


n m 
Di = 3 by (1.27) 
=1 


the vectors X*, yY*are solutions of the corresponding problems. 


Proof. According to Lemma 1.1, for any program X=(x,, x,,...,%*,) of 
problem (1.1)—(1.3), 


Hence, applying the conditions of Lemma 1.2, we obtain 
¥ cjx,<D ct. (1.28) 
f=. {=1 


Inequality (1.28), which holds for any feasible program X of problem (1.1)- 
(1.3), shows that X* is indeed the optimal program. 

The optimality of program Y°* of the dual problem (1.4)-(1.5) is establish- 
ed analogously. This completes the proof. 

Lemma 1.2 can be interpreted economically thus. If for some feasible 
scheduling program of production modes X* and some tentative cost vector 
of production factors Y* the cost of the output commodity is equal to the 
overall value of the resources available, X* and Y*® are, respectively, the 
optimal production program and the optimal cost vector. 

Lemma 1.2 establishes sufficiency of conditions (1.27) for optimality of 
programs X*and Y¥*. Below we shall see that equality (1.27) is also a ne- 
cessary condition for the optimality of programs X* and Y*. 

Lemma 1,3. If the linear form (1.4) of the dual problem (1.4)-(1.5) is 
unbounded from below on the set of its feasible programs, the primal problem 
(1.1)-(1.3) has no feasible programs. 


Proof. By assumption, there exists a set of feasible programs {Y,} of the 
dual problem (1,4)-(1.5) such that 


lim (B, Y,)=— 00. (1.29) 
ko @ 


If we assume that the primal problem (1.1)-(1.3) has a feasible program 
X, then according to Lemma 1.1 


(C, X)<(B, Y,) 


for any natural &. Passing to the limit as k—-oo and applying (1.29), we 


obtain 
(C, X)=— oo. (1.30) 


If all the components of the vector X are finite, equality (1.30) is meaning- 
less. 

Hence the assumption that there exists at least one feasible program of 
problem (1,1)-(1.3) is false. This completes the proof. 

We now elucidate the geometrical meaning of Lemma 1.3 in the (m+ 1)- 
dimensional space of points 


U= (u,, uy, ene9 to): 


If there exists a set of hyperplanes passing through the origin, situated 
above the cone X, and intersecting the Q-axis at points whose (m+1)-th 
coordinate tends to —oo, the line Q and the cone K are disjoint. 
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Lemma 1.3 proves useful when it is necessary to establish the solvability 
of a linear-programming problem. 

All three lemmas given above were formulated for a canonical linear- 
programming problem. The same propositions hold, obviously, also for 
problems with homogeneous restraints since in constructing the dual prob- 
lem of problem (1.17)-(1.19) we reduced the primal problem to canonical 
form. Furthermore, this property has been used implicitly in our economic 
illustration of the propositions. 


§ 2. On some properties of convex polyhedral cones 


2-1, In the proof of duality theorems in the next section certain proper- 
ties of convex polyhedral cones are used. These properties will be estab- 
lished in the present section. In the following, when speaking of points or 
vectors we shall refer to elements of an a-dimensional vector space, n 
being arbitrary. 

In Chapter 2 we advanced two equivalent definitions of a convex polyhedral 
cone in an a-dimensional space (see 3-5), 

Here it is more convenient to operate with the second definition, which 
is as follows. 

The set Kp, comprising the points 


N 
P=> BR +P, 
é=1 


where 6,;>0, /=1, 2,...,N and R,R,,...,Ry, P,are some points (vectors) of 
the n-dimensional space, is called a convex polyhedral cone with apex at P, 
spanned (generated) by the vectors R,, R,, ..., Ry. 

We give one elementary property of convex cones (for the corresponding 
definition, see Appendix, 3-2). 

Lemma 2,1. Let ll be the support hyperplane of the cone T at the point 
P. Then this hyperplane contains any point P’ of the form 


P’=P,+p(P—P,), (2.1) 
where P, is the apex of T, and u>0. 
Proof, Let the equation of the hyperplane II be 
(A, X)=c. (2.2) 


To be specific, assume that p» in (2.1) is larger than 1 (the case p<1 can 
be dealt with analogously). 

If P’ is any point of the cone T which may be represented by (2.1) for 
w<il, there exists a number S,0<S<1, such that 


P=SP’ +(1—S)P’. 
By assumption 


(A, P)=c. 
Hence 
(A, SP’ +(1—S) P’) =, 
or, after obvious manipulations, 
S[(A, P’)— ce] +(1—S) [(A, P”)—c] =0. (2.3) 
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Since H is a support hyperplane of the cone T, each of the elements in 
brackets in (2.3) is nonpositive (or nonnegative). Therefore, for equality 
(2.3) to hold for 0<S<1 it is necessary that both brackets vanish, i.e., 


(A, P’) =(A, P’)=c. 


Thus, any point P’ located on a ray issuing from P, in the direction of P 
belongs to the support hyperplane HI. *This completes the proof. 

Applying Lemma 2,1 for p=0, we obtain 

Corollary 2.1. Any supporthyperplane of a convex cone contains its apex. 

Lemma 2.1 and its corollary apply to any convex cone. In particular, 
they hold for convex polyhedral cones. 

2-2, The propositions dealt with below hoid only for convex polyhedral 
cones. 

Consider a convex polyhedral cone K spanned by the vectors R,, R,, ..., Ry 
with apex at the origin. Let the support hyperplane of the cone K at the 
point Q€K be denoted by Ilo. 

Lemma 2.2, For any boundary point P of the cone K there exists e>0 
such that any support hyperplane Mg, where |P—Q|<e*, contains the point P. 


Proof, Let us assume the contrary, i.e., there exists a sequence {Q,} 
of boundary points of the cone K such that 


lim | P— Q,|=90, 
a> @ 


but none of the hyperplanes Hg, contains the point P. 


Let 
Sa 
Q=3 BM Ri, 
k=1 
where Bp) >0, k=1,2,...,8, There is an infinite number of points Q. The 
number of different systems comprising the vectors R,, R,,..., Ry is a priori 


finite. Therefore, from the sequence ({Q,} we can find an infinite subsequence 
{Qc,} = {Q,} such that 


where the system of vectors R;,, Ri,, ...,R:,is the same for all the points 
Qu,=9,, Be’ >0, k=1, 2, ..., s,f#=1,2... We have thus formed a sequence {Q,} 
of boundary points of the cone K with the following properties: 

(a) for any 4, 


Aw alt 
Q=> Be Ries (2,4) 
k=1 
where the system of vectors R,, R,,,..., Ri, is independent of the point Q,, 
and 
Be >0 fork=1,2,...,5; tA 52; eke 


(b) lim | Q, — P| = 0; 
(c) PETS, forany t=1,2,... 


We shall show that such a sequence cannot exist. 


* We remind the reader that the symbol | A| denotes the length of vector A: |A| =Y(A, A). 
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Let K denote a polyhedral cone spanned by the vectors R;,, Ri -.-, Ri, with 
apex at the origin. According to (a), 


Q,€K, t=1, 2,... 


Condition (b) indicates that P is the cluster point of the sequence {Q,}. Since 


the convex polyhedral cone K is closed (see Chapter 2, Lemma 3.3), PEK. 
Hence, 


P=> BRin (2.5) 


where 6, 0, k=1, 2, 
Consider an eae point Q, of the sequence {Q,}. Let the equation of 
the support hyperplane I>, be 
(Ay X)= 0, (2.6) 


To obtain equation (2.6) we applied Corollary 2.1 according to which the 
hyperplane Iz , contains the point 0, which is the apex of the cone K. 
Allowing for representation (2.4) of the point Q, € Il, , we have 


(Ay, QD=D BL (Ap Ry) =0. (2.7) 
k=1 


Since (Aj, Rji,) <0 (Ig, is a support hyperplane of the cone K), and £,>0, equa- 
lity (2.7) applies only if 


(Ay, R;,) = 0, k=1, 2, eet, Ss. (2.8) 
Relationships (2.5) and (2.8) yield 


(Ay, P)= 


which indicates that P€ No,- 

None of the points of the sequence {Q,} satisfying (a) and (b) can satisfy 
(c). This contradiction establishes the proposition of Lemma 2.2. 

2-3, Let Tp, be a polyhedral cone with apex at the point P spanned by 
nonzero vectors T,,T,, ..., Ty,» 

We shall assume the polyhedral cone T, to satisfy the following condition: 

For any y,>0,; 


N 
3 HiT =0 
on =i 
only i 
Y=, oe Hy, = 0. 


It can be shown (see Exercise 3) that this condition is equivalent to the re- 
quirement that the apex P of the cone T, be its extreme point. 

Lemma 2.3. If the intersection of convex polyhedral cones K and Tp 
contains only one point P, there exists a hyperplane "1 with equation 


(A, X)=0, 


which is the support hyperplane of K at the point P((A, X)<0 tf XEK; (A, P)=0) 
such that 
(A, Q>0 (2.9) 


for all the points Q of the cone Ty, except tts apex P. 
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Proof, Consider the set T&(0<8<1) comprising the points 


Ny 
Q= 6,T,+PET>, 
such that = 


N 
=1 


It can easily be verified that Tf is a closed bounded convex set. 
The proof is left to the reader (see Exercise 4). 


N 
The point P¢7P) Indeed, if 0<d<>'s,, then according to the property, 
k=1 


given above, of the cone Tp 
N 
D3 Bp 7, #0, 
k=) 


so that any point P’of the set 7)’ has the form 


P’=P+P’, 
where P"+#0,i.e., P’+P. 

The sets K and 7%) are thus disjoint. Let PB be the point of the cone K 
least removed from the set 7. The existence of such a point follows from 
the fact that the set K is closed and the set Tf) is bounded (see Appendix, 
3-3). P, is obviously a boundary point of the cone K. Let II, be the support 
hyperplane of K at the point P, such that the set 7 lies in the half-space 
generated by II, and does not contain K. (Such a point indeed exists; see 
Appendix, Corollary 3.2.) 

We shall show that 

lim| Py—P| =0. (2.10) 


Indeed, otherwise we could form a sequence {P;,} converging to the point 
P+P (we use here the boundedness of the set of points P,, 0<6<1). 
Obviously, Py=P+6T,€ TP so that 


| Ps—P|=8)7,|. 

Hence, by definition of P,, there exists a point P,é€T such that 
|P,—P, |< 8) 7, |. 

Passing in this equality to the limit as 6,—~-0, k=1, 2, ..., we have 


lim Ps, = lim Ps, = P, 
k+@ kw 
It follows from this relationship and from the fact that the sets K and Tp 
are closed that the point P belongs to the intersection of the cones K and 
Tp(by assumption, P,€7p, P,€K). But the intersection of K and 7p contains 
only one point P, The ensuing contradiction proves the validity of (2.10). 
Let us now take 6=6*>0so small that 


| P—P5| <e, 


where e is the quantity entering the conditions of Lemma 2.2. According 
to this lemma, IIg* is the support hyperplane of the cone K at the point P. 
Assume the hyperplane Ikke to have an equation of the form 


(A, X) = 0. 
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From the definition of the hyperplane IIs. we then have 


(A, X)<c0 for XEK, (2.11) 
(A, P)=0, (2.12) 
(A, X)>0 for XETP”. (2.13) 


Let Q be an arbitrary point of the cone 7,, other than P. 
Then, obviously, there exists a number yw>0 such that 


Q,=P+H(Q—P)ETP”. 
Hence, applying (2.12) and (2.1 3) we have 


Thus, 
(A, Q)>0 (2.14) 


for all Q € T, other than P, Relationships (2.11), (2.12), and (2.14) indi- 
cate that the hyperplane IIs. satisfies all the requirements of Lemma 2.3. 
This completes the proof, 

Lemma 2.3 is often used in proving the duality theorems in the next sec- 
tion. In the proof of this lemma it is assumed that the convex cones K and 
T,are polyhedral, and P is the point (i.e. , the extreme point) of the cone 
Tp- Many examples can be given in which departure from these assump- 
tions as regards the cone K and the point P render the proposition of the 
lemma, generally speaking, invalid. As regards the cone 7,, it need not be 
polyhedral for Lemma 2.3 to hold. 

The proof of these propositions is left tothe reader (see Exercises 5 and6). 


§ 3. Duality theorems 


3-1, We have shown in §1 that either linear-programming problem with 
homogeneous restraints (1.17)-(1.19) and (1.20)-(1.22) is dual with respect 
tothe other. These two problems thus constitute a dual (conjugate) pair 
of linear-programming problems. Conjugate linear-programming prob- 
lems have certain interesting and important properties which place them in 
one class rather significant in applications. Some elementary properties 
of dual problems were considered in the end of §1. 

In this section we formulate and prove two principal duality theorems and 
derive from them some useful corollaries. The proof of the two theorems 
is based on properties of convex polyhedral cones which have been dis- 
cussed in the preceding section. The duality theorems are proved here for 
canonical linear-programming problems with homogeneous restraints. 

The general case is considered in the next section, where the concept of 
duality is extended to linear-programming problems given in arbitrary form. 

3-2, Consider a general linear-programming problem in canonical 
form (problem (1.1)-(1.3)). Problem (1.1)-—(1.3) and its dual problem (1.4)= 
(1.5) have the following important property: 

Theorem 3.1. If problem (1.1)-(1.3) has an optimal program, problem 
(1.4)-(1.5) is also solvable. Moreover, for any optimal programs X= (x,, 
Xe, eve, X,) ANd Y=(yy, Yur «+++ Yq) Of Problems (1.1)-(1.3) and (1.4)-(1.5), 
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respectively, the equality 


D» C,X, = » by (3.1) 


holds. 
Proof.1. The proof is based on the following geometrical considerations 


(see 1-2). Consider a convex polyhedral cone K spanned by the augmented 
restraint vectors of the primal problem, The geometrical image of the op- 
timal program of problem (1.1)-(1.3) is the "highest" point of intersection 
of line Q and cone XK (the point 8). Applying Lemma 2.3 to the cone K and 
to the ray issuing upwards along the line B from the point Q, we construct 
the support hyperplane I* of Kat the point B such that the vector e,,, 
drawn from II*is above 8. This hyperplane corresponds to the optimal 
program of the dual problem. 

2. We now proceed with a rigorous proof of the theorem. As usual, let 
A,be the j-th augmented restraint vector of problem (1.1)-(1.3): 


A,;=(4,,, ae), eo es aa fr c)). 


Let further 7 
B=(5,, b,, re b.; A)‘, 


where B=(b,, b,, ..., 6,)7 is the constraint vector of problem (1.1)-(1.3), 
and A is the maximum of the linear form (according to the conditions of the 
theorem, problem (1.1)-(1.3) is solvable). 


Let X*=(x,, x;, ..., x%,) be the optimal program of problem (1.1)=-(1.3 ). In 
this case, according to the definition of the vectors Ay, j=1, 2, ..., 2, and B, 
= R  » 


= 


Let K denote the convex polyhedral cone with its apex at the origin spanned 
by the vectors A4,, s=1, 2,...,a. Relationship (3.2) indicates that BEK. 
Consider the ray S issuing from the point 8 and parallel to the direction 
vector e,,,=(0, 0, ..., 0, 1)% It is obvious that any point of the ray S, other 
than its end point B, lies outside the cone K. Indeed, if for some u>0d the 
point B+ pe,,, belongs to the cone K, there exist nonnegative numbers x,, 
jJ=1, 2, ..., a, such that 


B+ Mane 2 A,X. (3.3) 


It follows from the vector equality (3.3) that X=(x,, x,,.., x,)is a feasible 
program of problem (1.1)-(1.3) and that 


3) ot AAD, (3.4) 


Relationship (3.4) contradicts the assumption that A is the maximum of the 
linear form (1.1) subject to the conditions (1.2), (1.3). Consequently, equa- 
lity (3.3) is invalid, i.e. , for A>0 


B+Ae,,,¢K. 

All the points of the ray S, except for the end point 8B, thus lie outside K. 
The ray S may be considered as a convex polyhedral cone spanned by 
the single vector e,,, with its apex at the point B. The apex 8 of the cone 

S is obviously its point. 
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Lemma 2,3 thus applies to cone K and to ray S. In this case N,=1; P=B; 
T,= gsi; Tp=S. According to Lemma 2,3, there exists a hyperplane I* with 
equation 


(A, U)=0, 
such that 
(A, A) <0, J=1, 2, 0... m (3.5) 
(A, B+e.,,)=(A, €n4,)>0. (3.7) 


Condition (3.5) follows from the fact that II* is a support hyperplane of 
the cone K. Equality (3.6) indicates that hyperplane II* contains the point 
B, the apex of the cone S. Relationship (3.7 ) follows from condition (2.9) 
of Lemma 2.3. 

3, We may now easily establish the optimal program of the dual program 
(1.4)-(1.5). By virtue of (3.7), the (m+ 1)-th component of the vector A=(A,, 
Nar seer Ans Ama.) IS positive: 


(A, Coe) =A 54,0. 


Applying this fact, we define an m-dimensional vector Y*=(yi, Ys, eos, Ym)s 
taking 

* A, , 

y= baad 2,3 ees I: 


Rewriting condition (3.5) in an obviously equivalent form, we have 


(A, A) = N@ij+ Aa ail/S 0, 


or 


(ime) a= By" =1,2 
p> m+ Ai7= 2 a, Cy JH Ny 2y weg TE 


The vector Y* thus satisfies conditions (1.5) and as such is a feasible pro- 
gram of the dual problem (1.4)-—(1.5). 

We now prove the optimality of program Y* Expressing the scalar pro- 
duct in the left-hand side of equality (3.6) in terms of the coordinates of 
the vectors A and 8, we obtain 


m 
a Ab, +AneiA = 0, 
or = 


PT 6, = A. 
According to the (m+ 1)-the component of the vector equality (3.2) 
"% ‘ ak 
put cf 
Hence, 
R x m ‘ 
pti 67> DI b= A, (3.8) 
=1 st 


According to Lemma 1.2 equality (3.8) proves the optimality of program Y* 
of problem (1.4)-(1.5). 

The dual problem is thus solvable and its optimal program Y* satisfies 
equality (3.8). 

If, now, X and Y are two solutions of problems (1.1)-(1.3) and (1.4)—(1.5), 
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respectively, we have 


so that 


n m 
p> x j0;= Ly idi- 


This completes the proof. 

From this theorem follows the so-called first duality theorem 
applicable to problems with homogeneous restraints. 

Theorem 3.2. (First duality theorem.) If either problem of a dual pair 
(1.17) -(1.19) and (1.20)-(1.22) is solvable, the other problem ts also 
solvable. Any two optimal programs X and Y of these problems satisfy 
equality (3.1). 

Proof. Consider an arbitrary problem of a given dual pair. In §1 we 
showed that if it is reduced to canonical form and then its dual problem is 
formulated, we obtain the second problem of the dual pair. Therefore, to 
prove Theorem 3.2 it suffices to refer to Theorem 3.1. 

We observe that the first duality theorem can also be proved without 
applying the comparatively complicated Lemma 2.3 (see Exercise 9). 

3-3. We give some corollaries of the first duality theorem. 


Corollary 3.1. One of the problems of a dual pair (1.17)-(1.19) and 
(1.20) -(1.22) will be solvable if each of these problems has at least one 
feasible program. This condition is necessary and sufficient. 

Proof. 1. Sufficiency is established without referring to the first duality 
theorem. 

Let Y’=(y;, ys «++; Ym)be a feasible program of problem (1.20)—(1.22). In 
this case we have, from Lemma 1.1 for any feasible program X=(x,, x,, 

., x,) of problem (1.17)-(1.19), 


nr 


p2 ¢%/= % by; e 


Thus, the set of feasible programs of problem (1.17)-(1.19) is nonempty and 
the linear form (1.17) is bounded from above on this set. Therefore, accord- 
ing to Chapter 2, Theorem 4.4, problem (1.17)—(1.19) is solvable. 

The solvability of problem (1.20)-(1.22) is established analogously. 

2. Necessity follows from the first duality theorem. Indeed, if one of 
the problems of the dual pair is solvable the other problem is solvable too. 
Hence, each of the two dual problems has at least one program. 

Corollary 3.2. One of the problems of a dual pair has feasible programs 
and the set of feasible programs of the other problem is empty if and only 
if the linear form of the first problem is unbounded on the set of its feasible 
programs. 

Proof. 1. Sufficiency is given explicitly by Lemma 1.3. 

2. To establish necessity, we refer to the first duality theorem. If the 
set of feasible programs of one of the two dual problems is empty, this 
problem is unsolvable and, consequently, the other problem has no solu- 
tions either. By assumption, however, the latter has feasible programs. 
Hence, its unsolvability is due to the unboundedness of the linear form in 
the set of feasible programs (see Chapter 2, Theorem 4.4). 
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Dual pairs fall into the following three disjoint classes: 
(a) the two problems have feasible programs; 
(b) only one of the problems has feasible programs; 


(c) for each problem of the dual pair the set of feasible programs is 
empty. 
We give some illustrative examples. 
Example 1. The primal problem I’: 
5x, + x%,— max; 
X,+2x, <4, 
2x, +3x, <3, 
x,=>0, x,>0. 
The dual problem II’: 
4y, +3y,—min; 
y, + 2y, S85, 
2y,+3y, 21, 
9,0, y= 0. 
Each of these problems has, obviously, feasible programs: 
the vector (0, 0) is a feasible program of problem I’; 
the vector (1, 2) is a feasible program of problem IJ’ 
Example 2. The primal problem I’: 
5x, + *%,— max; 
X,— 2x, &2, 
x,—3x, <3, 
x, 20, x, 20. 
The dual problem II”: 
2y, +3y,— min; 
yty, 5, 
—2y,—3y, 21, 
¥, 20, y, > 0. 
The vector (0, 0) is a feasible program of problem I°. Problem IT’ has no feasible programs. Indeed, 
multiplying the first restraint of problem II° by 2 and adding it to the second restraint, we obtain 


—y, 211, 
or 


A= =, 


which contradicts the last restraint of the problem: y, 20. 
Example 3. The primal problem I’’’: 


5x, x, — max; 
X,—-*%, <1, 

—x,+%,< —2, 

Xx, 20, x, >0. 


The dual problem II’’’ 
¥,—2y,— min; 
Y—¥, = 5, 
—Yt¥.21, 
WH =0, Y2 = 0. 


Adding the first two restraints of problem I’’’ we have 
0< —!. 

The ensuing contradiction shows that problem 1’’’ has no feasible programs. Proceeding similarly with 
restraints of problem II’’, we obtain 0586. Hence, problem II’’’has no feasible programs either. 

Thus, each of the three possibilities (a), (b), and (c) exists in practice. 

It follows from Corollary 3.1 that the conditions of (a) are equivalent to 
the assumption that the two problems of the dual pair are solvable. Re- 
ferring to Corollary 3.2, we observe that (b) is tantamount to the require- 
ment that the linear form of one of the problems be unbounded in the set of 
its feasible programs. 
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We now deal with another corollary of the first duality theorem which 
gives the necessary and sufficient conditions for optimality of programs of 
conjugate problems. 

Corollary 3.3. The programs X*=(x,, x3,..., x4) and Y*=(y;, yr, 0+ Ym) OF 
problems (1.17)-(1.19) and (1.20)-(1.22), vespectively, ave optimal tf and 


only if Z is 
p> Cj =2 b: yi. 


Proof. Sufficiency follows from Lemma 1.2. Necessity is given expli- 
citly in the second part of the first duality theorem. 

3-4. For the following we shall require some new definitions. 

Consider the inequality restraints of problems (1.1)-(1.3) and (1.4)-(1.5). 
These are the inequalities (1.3) of the primal problem and the relationships 
(1.5) of the dual eee in 

The j-th restraint in (2 a;,y;2c,) is dual with respect to the j-th 


restraint in (1.3) hit 
The /-th restraint in (1.3) (or (1.5))is termed fixed if for any optimal 
program X*(¥*)of problem (1.1)—(1.3) (problem (1.4)-(1.5)) 


m 
xj =O (2 aii = Cy), 


i.e., if this restraint is reduced to an equality by any optimal program of 
the corresponding problem. 

The j-th restraint in (1.3) (or (1.5)) is termed free iffor at least 
one optimal program xX*(Y*) of problem (1.1)-—(1.3) (problem (1.4)-(1.5)) 


m 
xj >0 (Wai >e;), 
=1 


i.e. , if this condition is reduced to a strict inequality for at least one 
optimal program of the corresponding problem. 

The following theorem establishes a relationship between the restraints 
(1.3) and the dual restraints (1.5). 

Assume that problem (1.1)-(1.3) and, consequently, also the dual prob- 
lem (1.4)-(1.5) are solvable. 

Theorem 3.3. If some restraint in (1.3) is free (fixed), the dual restraint 
in (1.5) ts fixed (free). 

Proof. We shall use the notations introduced in the proof of Theorem 3.1. 

1. We first show that if the j,-th restraint in (1.3) is free, the /-th re- 
straint in (1.5) is fixed. 

We may reason geometrically thus: the optimal program of the dual 
problem corresponds to a hyperplane II* passing through the highest point 
of intersection of the cone K and the line Q (the point B) and lying above 
the cone K. 

Since the /,-th restraint in (1.3) is free, the vector B may be represent - 
ed as a nonnegative linear combination of the vectors A; the vector A, ap- 
pearing in the combination with a positive coefficient. The vector A, 
therefore belongs to the hyperplane II*, which, geometrically, indicates 
that the /,-the restraint in (1.5) is fixed. 

2. We now prove the first part of the theorem analytically. Consider an 
arbitrary solution 


Y* =(ys, Yoyo ees Ym) 
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of the dual problem (1.4)-(1.5). If we take 


A=(—y;, —Yny soey —Ym: 1), 
then 


(A, A,) <0, jJ=1, 2, 20 A, (3.9) 


since Y* is a program of problem (1.4)—(1.5). 
By assumption, there exists a solution 


X* = (<1, X35, 0205 Xn) 


of problem (1.1)-(1.3) such that xj,>0. Since Y*and X* are optimal pro- 
grams, we have (Corollary 3.3) 


A= p2 CX] = 2 by 
or, equivalently, 
(A, B)=0. (3.10) 
Substituting (3.2) into (3.10), we obtain: 


2 xj (A, A,)=0. (3.11) 


According to (3.9) each term in the left-hand side of (3.11) is negative, 
thus (3.11) is satisfied only if all the terms vanish. In particular, 


x), (A, A,) =0. 


By assumption, however, x;,>0. Hence 


(A, A, =— Dieu, +e, =0. (3.12) 

Since (3.12) applies for any optimal program Y*of problem (1.4)-(1.5), 
the /-th restraint in (1.5) is obviously fixed. 

3. We now prove the second part of the theorem: if the ,s-th restraint in 
(1.3) is fixed, the j-th restraint in (1.5) is free. 

Again, we first give a geometrical proof. From point B— the geometrical 
image of the solution of problem (1.1)—(1.3)— draw two vectors: the vector 
emai directed upward along the line Q, and the vector B—A,, drawn from 
A, to B. According to the conditions of the theorem, the cone spanned by 

and B—A , With apex at B satisfies all the requirements of Lemma 2.3. 
Therefore, through B we can draw a support hyperplane HI* of K such that 
all the points of the cone, except the apex 8, are above II*. The hyperplane 
II* is the geometrical image of the optimal program of problem (1.4)—(1.5) 
which reduces the j/,-th restraint in (1.5) to a strict inequality. 

4. We now give a rigorous proof of the second part of the theorem. Let 
the s,-th restraint in (1.3) be fixed. By definition, the /,-th component in 
each optimal program of problem (1.1)-(1.3) is zero. In other words, the 
vector A, does not enter any of the representations (3.2) of the vector 8B. 

Consider a convex polyhedral cone Ts with apex at B spanned by the 
vectors e,,, and B—A,.. Let 


R=B+p, (B—A,) + tens (3.13) 


yw, 20, p,=0be a point of Tz. We shall show that when p,+p,>0, REX. For 
p,=0 this was established in the proof of Theorem 3.1. 
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Now let p,>0. Assuming the contrary, 


R=B-+p, (B—A,) + Hrs S K, 


i,e., 
R= > x/A, (3.14) 
where x, 20 for j=1, 2,..., 2. Take the vector 
r_ by | l 
R Tae “et en, R, (3.15) 


Since the cone K is a convex set, R’EK 
Substituting (3.13) into (3.15), we obtain 


R=B+ Pe ean EK. 


If p,>0, this relationship iscontradictory to the assumption that A (the 
(m-+1)-th component of the vector 8B) is the maximum of the linear form (1.1) 
subject to the conditions (1.2) and (1.3) (see proof of Theorem 3.1). If p,=0, 
however, 

B=R att Ay 

B= R= Tee ett 
whence, applying (3.14), we obtain (3.2) containing A, with a positive coef- 
ficient, which is a priori impossible, 

Assumption (3.14) is thus false. Hence, R€X if the coefficients p,>0 
and p,=20 appearing in (3.13) satisfy the conditions 


By +H,>9. 


Thus, all the points of the cone Tg except the apex B lie outside the 
cone K. 

Observe that 8 is the point of cone Tg. Indeed, for any p,>0, p,>0 it 
follows from the condition p,+p,>0 that 


R= B+ Ht, (B—A,)+Pem+s +Be K, 
Therefore ie 
Bb, (B—A,.) + been 4 19. 


5. All the aforegoing justifies the application of Lemma 2.3 to cones K 
and Tg. In this case 


P=B: N, = 2; T,=B—A,, T, =e 


m+1° 


According to Lemma 2.3, there exists a hyperplane I* with the equation 


(A, U)=0, 
satisfying the conditions 
(A, A)<0, j=l, 2,...,m (3.16) 
(A, B)=0; ~ (3.17) 
(A, B+en4,)=(A, Cn y,)>03 (3.18) 
(A, B+ (B—A,,))=— (A, A,,)>0. (3.19) 


Inequalities (3.16) and equality (3.17) indicate that II* is the support 
hyperplane of the cone K at the point B. Relationships (3.18) and (3.19) 
follow from that property of the hyperplane II* according to which all the 
points of the cone Tg, except the apex, are above II*. 
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According to (3.18) the (m+ 1)-th component of the vector A=(A,, Ay, oo«r Aw 


Aws,) iS positive. We may thus introduce a vector Y*=(y;, ys, ..., ym) taking 
¢ ry 
aaa wa $ i=1, Bee ag m. 


Applying conditions (3.16) and (3.17) we conclude, as in the proof of Theo- 
rem 3.1, that the vector Y* is a solution of the dual problem (1.4)-—(1.5). 
According to inequality (3.19) 


m 
2 M121 jg t hem 18 jg <9 
or, equivalently, 
Ai 2 
x (- Am+ ) ai, = De yi), >C, (3.20) 
f=1 were i= 


We have thus derived an optimal program Y* of problem (1.4)-(1.5) satis- 
fying inequality (3.20). By definition, this indicates that the /,-th restraint 
in (1.5) is free. This completes the proof. 

Consider a linear-programming problem with homogeneous restraints 
(1.17)-(1.19) and the dual problem (1.20)-(1.22). 

We reduce problem (1.17)-—(1.19) to canonical form introducing additional 
variables 


Xyy(=b;— Qoa,x,0, i=1,2,...,m, 
{=i 

We may now extend the definitions introduced at the beginning of this section 

to problems with homogeneous restraints. In this case the restraint vec- 

tors A, have the form; 


A (2, », Qj a er if 1<f/<n 
7 \ (0, ...,0,1,0,...,0)7, if z+l<jans+om. 
n-j 


Therefore the restraint 
m 
(Ay Y= Laiyiee, (Uj=1,2,..., a) 


of problem (1.20)—(1.22) is dual with respect to the restraint x,>0of prob- 
lem (1.17)-—(1.19). Analogously, the restraint 


(Angy Y=, 20 (é=1, 2,.... m) 


of problem (1.20)-(1.22) is dual with respect to the restraint 


n 
¥n4i=b)— 2 ai,x;>0 
of problem (1.17)-—(1.19). 

The j-th restraint in (1.21) is thus dual with respect to the /-th restraint 
in (1.19); the é-th restraint in (1.22) is dual with respect to the i-th re- 
straint in (1.18). 

If problem (1.20)-—(1.22) is taken as the primal problem then, as shown 
in §1, problem (1.17)-(1.19) is the dual problem. Hence, the j-th restraint 
in (1.19) (the i-th restraint in (1.18)) is dual with respect to the j-th re- 
straint in (1.21) (i-th restraint in (1.22)). The j-th restraints in (1.19), (1.21) 


x, 20, Fie, (3.21) 


97 


and the i-th restraints in (1.18), (1.22) 
| 
21; ,%,<b;, yO (3.22) 
{/=1 


are thus pairs of dual restraints of the conjugate problems (1.17)- 
(1.19) and (1.20)-(1.22). Observe that the dual pair (3.21) corresponds to 
the j-th column in Table 3.1, and its component restraints are therefore 
termed column restraints. 

Analogously, the components of the dual pair (3.22) are termed row 
restraints (they correspond to the i-th row of Table 3.1). 

We now extend the definition of the fixed and free restraints given at the 
beginning of this section to problems with homogeneous restraints. A 
restraint is said to be free if there exists a solution reducing the restraint 
to a strict inequality; a restraint is said to be fixed if it is reduced to an 
equality for all the solutions of the given problem. 

After these preliminary remarks we can formulate the second theorem 
for problems with homogeneous restraints. 

Theorem 3.4. (Second duality theorem.) If the conjugaie problems (1.17)- 
(1.19) and (1.20)-(1.22) are solvable, then in each pair of their dual re- 
straints (either row or column) one restraint is free and the other is fixed. 

The second duality theorem is an obvious consequence of Theorem 3.3 
and of the definitions for problems with homogeneous restraints. 

3-5. The first and the second duality theorems will be used repeatedly 
in the following. It is therefore important that their actual content be com- 
pletely understood. The propositions of the duality theorems become clear 
if they are restated in terms of production of a homogeneous commodity. 
These terms have already been used inthe economic interpretation of the 
pair of dual problems with identical restraints. 

According to the first duality theorem, an optimal production program 
exists only if all the production factors have evaluations. Suppose that the 
production has an optimal program and, consequently, all the factors can 
be evaluated. In reality this is not always so. Both the optimal production 
program and the system of costing of the production factors are not 
usually uniquely determinable. 

According to the second part of the first duality theorem, the cost of the 
commodity produced under some optimal production program is equal to 
the total cost of the resources available, regardless of the actual evaluation 
of the production factors (the components of the solution of the dual prob- 
lem). Thus, a characteristic property of the optimal program is that it 
ensures that the cost evaluation of the commodity produced and that of the 
resources consumed in production are equal. With any other program of 
utilization of the technological means (that is, other than optimal} the pro- 
duction process will be deficient: the cost of the commodity produced will 
be less than the total cost of the resources available. This is due to 
the fact that under a nonoptimal program the possibilities of production are 
not fully exploited. 

We recall that a cost vector of production factors is taken as the 
set of preliminary evaluations of production factors (a program of the dual 
problem) which minimizes the total evaluation of the resources available 
(the linear form of the dual problem), The meaning of this now becomes 
clear, When this condition is observed, the characteristic property of the 
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optimal program is fulfilled. If, however, the cost vector is taken 

as some set of preliminary evaluations not minimizing the total cost 
of the resources available, the characteristic property of the optimal pro- 
gram does not apply. 

In this case the cost of the commodity produced under any program, in- 
cluding an optimal program, is less than the total cost of the resour- 
ces available. The costs culminating in this situation are obviously 
undesirable. 

We shall now proceed with the economic interpretation of the second 
duality theorem. 

We first consider the row restraints of the dual pair of problems defining 
a given production process. 

Suppose that under one of the optimal programs of utilization of techno- 
logical facilities the i-th production factor is not fully exploited. Then, 
according to the second duality theorem, its cost vanishes. Conversely, 
if the cost of an i-th production factor (with any cost vector) is zero, 
there exists an optimal production program under which the resources 
of the i-th factor are not fully exploited. 

This situation can be regarded as quite natural. 

Indeed, a factor whose resources exceed the requirements (from the 
viewpoint of some optimal production program) is not critical in the produc- 
tion process: a certain reduction of the resources of this factor will not 
limit the scope of production. The cost of this factor from the view- 
point of the given production process can naturally be taken, therefore, as 
zero. Now suppose that the excess of the particular factor has been eli- 
minated. In this case the factor acquires a certain value: further reduc- 
tion of the resources will affect the volume of the output commodity. The 
cost of the factor thus becomes positive. 

We now give an economic interpretation of the column restraints of the 
dual pair of problems. 

According to the second duality theorem, a given production process is 
foreseen by some optimal program only if the resulting cost of the commo- 
dity produced is equal to the cost of the resources consumed (from the 
viewpoint of any cost vector). The intuitive content of this proposi- 
tion is quite obvious: there is no point in following a production process in 
which the costs incurred will exceed the potential income. 


§ 4. Linear-programming problems in arbitrary form 


4-1. In §3 we established duality relationships for linear-programming 
problems with homogeneous restraints. In this section we propose to ex- 
tend these results to linear-programming problems given in arbitrary form. 

We formulate a general problem of this type. 

Maximize the linear form 


subject to the conditions 
: <b; i=1, 2,..., m<m, (4,2) 
Box) =b, i=m +t, m,+2,..., m; 
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4,220, j=l, 2,..., aca. (4.3) 


One group of restraints (4.2) imposed on all the variables of the problem 
are inequalities (numbering m,), whereas the other group consists of equa- 
lities (the number of equalities being m—m,). Moreover, some of the vari- 
ables x, are assumed to be nonnegative (n, variables), The fact that it is 
the first m, restraints (4.2) that are inequalities and that constraints (4.3) 
are imposed on the first nz, variables does not detract from the generality 
of the statement of the problem. This arrangement can always be changed 
by suitably renumbering the variables and the restraints of the problem. 
The restraints of problem (4.1)-(4.3) are obviously mixed (as opposed to the 
homogeneous restraints of problem (1.17)-(1.19)). Problems of type (4.1)- 
(4.3) are therefore called problems with mixed restraints 

By definition, the dual (or the conjugate) problem with respect to prob- 
lem (4.1 )-(4.3) consists in minimizing the linear form 


PT (4.4) 
subject to the conditions 
ee =O, Joely 2) s40y Nee hy 
/ : (4.5) 
24 \ =Cy, J=n,t+1, 2,42, ..., 48 
y,20, i=l, 2,..., mam. (4.6) 


The dual problem of a problem with mixed restraints (4.1)-(4.3) is thus 
formed according to the following rules. 

If the variable x, of problem (4.1)-(4.3) is assumed nonnegative, the 
j-th restraint in (4.5) is an inequality. If, however, no such restraint is 
imposed on x,, the j-th relationship in (4.5) is an equality. Analogous cor- 
respondence is established between restraints (4.2) of problem (4.1 )-—(4.3) 
and constraints (4.6) of problem (4.4)-(4.6). If the i-th restraint in (4.2) is 
an inequality, y;=>0. Otherwise, the variable y,; may take on either positive 
or negative values. 

It can easily be verified (see Exercise 10) that problem (4.1 )+(4.3) is dual 
with respect to problem (4.4)=-(4.5). 

These two problems may therefore be designated asa pair of dual 
(or conjugate) problems. 

Observe that the preceding definition of a dual problem does not contra- 
dict the concept of duality introduced elsewhere for problems with homo- 
geneous restraints and for canonical problems. 

If we take m,=m, na,=n problem (4.1)-(4.3) reduces to a problem with 
homogeneous restraints (1.17)-(1.19), and the dual problem (4.4)-(4.6) 
transforms to problem (1.20)-(1.22). For m,=0, n,=n”, problems (4.1)—(4.3) 
and (4.4)-—(4.6) reduce to problems (1.1)-(1.3) and (1.4)-(1.5), respectively. 

4-2, Each linear-programming problem of type (4.1)—(4.3) will be as- 
sociated with the following problem with homogeneous restraints, . 

Maximize the linear form 


Revit 3 eimsinw) (4.7 
subject to the conditions ~ 
ry n 
242i + ZX 4, (Xj; —Xj+n,) <5, (4.8) 
V2 cag ih, 
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n 


ny é ee é 
—2 G; ;Xj— » ai, (%7—Xj+n,) a — 5, 
=1 


j=en,+ (4.8) 
i=m,+1, m+2,..., m; 
xj20, j=l, 2,..., 2, a+1,..., 242, (4.9) 


where a,=n—n,is the number of variables in problem (4,.1)-—(4.3) which are 
not assumed nonnegative. 

We establish a correspondence between problems (4.1 )-(4.3) and (4.7)— 
(4.9). 

An a-dimensional vector X=(x,, x, ..., x%,)and an (1+n2,)-dimensional vec - 
tor X’ =(x,, %, -.., Xnen,) Will be said to be corresponding if their com- 
ponents are related by the expressions 


{ * 8 te eee 
x<,>= 


: ; (4.10) 
Xj— Xjan, J=nti,a,+2,...,4. 


To any (n+n2,)-dimensional vector xX’ there obviously corresponds a unique 
n-dimensional vector X. Conversely, to any n-dimensional vector X there 
corresponds an entire family of (n+ .x,)-dimensional vectors X’ 

The correspondence established by (4.10) is thus single-valued in one 
direction only. 


Let X’ =(x, 4, ..., Xnan,) be a feasible program of problem (4.7)-—(4.9). 
Applying (4.10), we observe that the corresponding vector X=(x,, x,, ..., %,) 
is a feasible program of problem (4.1)-(4.3). 

Indeed, : : : 

> a; x; + p> ai, (xj — Xj+n,) = p> a; Xj (4.11) 
{=1 f=a,+1 fz 


Applying, further, conditions (4.8), we obtain 
n <b, i=1, 2, ...,m, 
2 ap*) >b, i=m,+1,..., m, 


a <& Dj, i=l, 2, ooeg My, 
a 
> iy*ys = b,, i=m,+1, m, +2, ee. M~ 


Moreover, with j=1, 2, ...,A,, %,=x;20. 
The vector X=(x,, x,,..., *,) thus satisfies restraints (4.2), (4.3) of prob- 
lem (4.1)-(4.3) and as such is a feasible program. 

Let now X=(x,, x,,..., *,) be a feasible program of problem (4.1)-(4.3). 
Among the (n+a,)-dimensional vectors X’ corresponding to X a priori 
there exist vectors with nonnegative components. One of these is the vec- 

tor X’= (xi, Xa, -++ » Xnen,) Where 


or 


cn ya Pe era oe 
in| max (0, %,)s j=Hn +1, n,+2, eee, R, (4,12) 
max(0,—x,), j=n+l,a+2,...,A+n,. 


The nonnegativity of all the components of X’ is obvious. The corres- 
pondence of the vectors X and X’ follows from the equalities 
Xj—Xj+n,= Max (0, ¥,)—-max (0, —x,) = 0—(—x)=x, if x,<0, 


which apply for j=n,+1, 2,+2, ..., &. 
It follows from (4.11) and from (4.2), (4.3) that any vector X’ with 
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nonnegative components corresponding to a feasible program X of problem 
(4.1)-(4.3) is a feasible program of problem (4.7)-(4.9). 

To sum up, to any feasible program of problem (4.7)-(4.9) corresponds 
a feasible program of problem (4.1)-(4.3). Conversely, to any feasible 
program of problem (4.1 )-(4.3) corresponds some set of feasible programs 
of problem (4.7)-(4.9). This correspondence is established by (4.10). 

We shall show that the linear forms of problems (4.1 )—(4.3) and (4.7)- 
(4.9) take on the same values on the corresponding programs X and X’ of 
these problems. : 

Indeed, applying (4.10), we obtain 

n n, n ; , 

» cj )= 2 eet 2 ey (x; —Xj+n,)- 
Hence, it follows that (4.10) also establishes a correspondence between the 
solutions of problems (4.1)—(4.3) and (4.7)-(4.9). 

Problem (4.4)—(4.6 ), the dual problem of (4.1)-(4.3), can easily be re- 
duced to the form (4.1)=—(4.3). It suffices to take c,=—Cy, aj,=— iz 6, =—);. 
With this substitution, problem (4.4)-—(4.6) reduces to maximization of the 
linear form 


subject to the conditions 


va ety Jl, 2) sae sR 

a = 

sf = Cy jJ=an, +1, sre yg A, 
y; = 9, i=1, 2, eee yg m,. 


To problem (4.4)-(4.6), therefore, corresponds the following problem 
with homogeneous restraints. 

Minimize the linear form 
ms m , ’ 
Debt 2b: Wi — Yams) (4.13) 
subject to the conditions “™ 
m m 
>> TEDL: + 2, Qi) (Yi —Yi4im,) = Ch 


tai 


J=1,2,..., 4%, 
Ms $ m : ; (4.14) 
—s 2 25 Mt — 2 (Yt —Jt4+m,) wey, 
Janie ly. c6i5, 
yi0, i= 1, 2, ..., m, m+i, ..., mom, (4,15) 


where m,=m—m, is the number of variables in problem (4.4)—(4.6) which 
are not constrained by the assumption of nonnegativity. 


According to (4.10), an m-dimensional vector Y=(y,, y,, .-., y,) and an 
(m +-m,)-dimensional vector ¥Y’=(y,, y:, .., Ym+m,) are said to correspond if 
Ms ad tae ee 
= ’ ’ , (4,16) 
Yi — Sir my t=m, +1, +o ey mM, 


We have proved that to any program Y’ of problem (4.13)-(4.15) corresponds 
a feasible program Y of problem (4,4)-(4.6). Conversely, any vector Y’ 
with nonnegative components corresponding to a feasible program Y of 
problem (4.4)-(4.6) is a feasible program of problem (4,13)-(4.15). More- 
over, if Y and Y' are two corresponding programs of problems (4.4)-(4.6) 
and (4.13)-(4.15), the optimality of either program yields the optimality of 
the other. 
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4-3, We are now ready to extend duality theorems to problems with 
mixed restraints. 

Theorem 4.1. (First duality theorem: general case.) If one of the prob- 
lems of a dual pair (4.1)-(4.3) and (4.4) -(4.6) ts solvable, the other prob- 


lem ts also solvable. Any optimal programs X=(x,, x,, ..., *,)and Y=(y,, 
Ys. «+01 Yu) Of these problems satisfy the equality 
2a erty = Do (4.17) 
Proof. Assume that problem (4.1)-(4.3) is solvable and let X be its 
optimal program. Consider the vector X' = (x, CoP wes Xn vas) associated with 


X by (4.10) and having nonnegative components. According to the above, X’ 
is a solution of (4.7)-—(4.9). 

We now apply the first duality theorem for problems with homogeneous 
restraints (Theorem 3.2) according to which problem (4.13)-(4.15), the dual 
of problem (4.7)-(4.9) is solvable and its optimal program Y’=(y,, ys, ..., Ymem) 
satisfies the equality 

m 


tt, n ms e , 
» cx Dy €,(X;—xj4n,) =Doyt LD oi(¥%—yi+m,)- 
{=1 j=n,+1 f=1 i= 


m,+1 


(4.18) 


As stated before, the vector Y corresponding by (4.16) to the optimal pro- 
gram Y’ of problem (4.13)-(4.15) is a solution of problem (4.4)—(4.6). 
Further, applying the equalities 


nr n n 
c C(X;—X jan.) = Dy CX), 
Dy it 2 j(*1—Xj+m) >» m1 


m, ; m ; , m 
ot DO (Yi — +m) =a OL; 
f=1 f=m,+) f= 


and (4.18) we obtain (4.17) which holds for any solutions X and Y of the 
dual pair. 

In our proof we set out from problem (4.1 )-(4.3), If now we assume sol- 
vability of problem (4.4)-(4.6), the proof of the theorem is analogous. How- 
ever we do not give the proof again since problem (4.1)-(4.3) is dual with 
respect to problem (4.4)-—(4.6). This completes the proof. 

The second duality theorem in the general case applies only to inequality 
restraints. A pair of restraints of the dual problems (4.1)-(4.3) and (4.4)- 
(4.6) with the same subscript /(1< jaa,) or the same subscript i(li qi<m,) 
will be called dual. Inthe first case (given /) the dual conditions are 
called column restraints, and in the second case (given i) they are 
said to be row restraints. 

The definition of free and fixed restraints for these problems is not dif- 
ferent from the corresponding definitions for problems with homogeneous 
restraints. We shall therefore not repeat these definitions. 


Theorem 4.2. (Second duality theorem: general case.) If the dual pair 
(4.1)-(4.3) and (4.4)-(4.6) is solvable, then in each pair of dual restraints 
(column restraints for j=1,2, ..., 2, and row restraints for i=1,2,...,m,) 
one restraint is free, and the other ts fixed. 

Proof. Consider problems (4.7)-(4.9) and (4.13)-(4.15) related to prob- 
lems (4.1)-(4.3) and (4.4)-—(4.6). Assume that the i-th restraint in (4.2) of 
problem (4.1)-(4.3) is free (i=1, 2,..., m,). In this case the i-th restraint 
in (4.8) of problem (4.7)-—(4.9) is also free. 

Indeed, let X be an optimal program of problem (4.1 )-(4.3) reducing the 
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i-threstraint in (4.2)toaninequality. If xX’is a vector with nonnegative com - 
ponents corresponding to program X according to (4.10), then, from the 
above, it is a solution of problem (4.7)-(4.9). Now, applying (4.11), we may 
assert that X’ reduces the i-th restraint in (4.8) to an inequality. Hence 

this is a free restraint. 

Now, let the i-th restraint in (4.2) be fixed. Considera solution X’ of 
problem (4.7)-(4.9). 

According to the above, the vector X corresponding to X’ by (4.10) is an 
optimal program of problem (4.1)-(4.3). Hence, the i-th restraint in (4.2) 
is reduced to an equality by vector X. Applying (4.11), we observe that 
the vector X’ reduces the i-th restraint in (4.8) to an equality. Our argument 
remains the same if row restraints are replaced by column restraints. 

To sum up, to a free (fixed) restraint of problem (4.1)-(4.3) corresponds 
a free (fixed) restraint of problem (4.7)-(4.9). 

The same correspondence between restraints obviously applies to prob- 
lems (4.4)~(4.6) and (4.13)—(4.15) also. 

After these preliminaries, the theorem is proved as follows. 

Let some inequality restraint of problem (4.1)-(4.3) be free (fixed). The 
corresponding restraint of problem (4.7)-(4.9) will have the same property. 
Applying the second duality theorem for problems with homogeneous re- 
straints (Theorem 3.4), we observe that the restraint of problem (4.13)-— 
(4.15), which is dual with respect to the restraint of problem (4.7)-(4.9) 
under consideration, is fixed (free). 

Further, applying the preceding correspondence between the restraints 
of problem (4.4)-(4.6) and (4.13)—(4.15), we conclude that the restraint of 
problem (4.4)—(4.6) dual with respect to any free (fixed) restraint of prob- 
lem (4.1)-(4.3) is fixed (free). This completes the proof. 

Thus, both duality theorems hold for linear-programming problems 
given in arbitrary form. 

All the corollaries of the duality theorems which were derived in the pre- 
ceding section for problems with homogeneous restraints and for canonical 
problems can, obviously, be extended to the general case, also. Further- 
more, all three lemmas of §1 can be extended without any modification to 
linear-programming problems with mixed restraints. The proofs in each 
case follow line by line the arguments given of §1. Inthe following we 
shall apply these lemmas to linear-programming problems in arbitrary 
form. 


§ 5. Optimality criteria and decision multipliers 


9-1, When solving linear-programming problems, it is most important 
that there be some means of testing the feasible programs for optimality. 
In other words, we must be able to answer whether a given program is 
optimal. 

The necessary and sufficient conditions for the optimality of a feasible 
program were first established by L. V. Kantorovich /61/. Later it was 
found that these conditions (generally called the optimality criterion of 
programs of a given problem) are closely connected with duality and are, 
essentially, consequences of the duality theorem. This section deals 
mainly with this connection. Such an approach enables us to apply the 
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results of preceding sections of this chapter and, in our opinion, makes 
the discussion more natural. 

Consider an arbitrary problem with mixed restraints (problem (4.1)— 
(4.3)). Following Kantorovich /61/, we shall refer to the parameters 4, 


Ay .--, dA, aSdecision multipliers*® if 
m 
(a) 2 ih > Cp J=1, 2,..., a3 (5.1) 
(b) 2 Ais = cp f=n,+1,2,42,...,% (5.2) 
(c) A; =0, i=1, 2, ..., m; (5.3) 
(d) for some feasible program X=(x,, x,, ..., *,) of problem (4.1)-(4.3), 
p> a,A,=c, for x,>0 (l<j<a,); (5.4) 
n 
4,=0 for 2 a,,x,<b; (i\<icm). (5.5) 
j=! 
The vector A=(A,, 4,, .-., 4,) Whose components are the decision multi- 


pliers 4; will be called the decision vector of problem (4.1)-(4.3) 
(the decision vector of program X). According to the following 
theorem, finding the decision vector is equivalent to solving problem (4.4)— 
(4.6), the dual of problem (4.1)—(4.3). 

Theorem 5.1. The set of all decision vectors of problem (4.1) -(4.3) 
coincides with the set of optimal programs of problem (4.4)-(4.6). 

Proof, 1. Let A=(A,, A,, ..:, 4,)be a decision vector ofproblem (4.1)—(4.3), 
related by (5.4) and (5.5) to program X=(x,, x,, ..., *,) of this problem. 

Conditions (5.1)-(5.3) satisfied by the vector A indicate that Aisa 
feasible program of problem (4.4)-—(4.6). The optimality of program A is 
thus obvious. 


Let £ denote the set of all subscripts /(/=1, 2,..., 2,) for which x,>0. 
Then : 
= : (5.6) 
2 CX, pa Fd a oh, C¥y 
Applying equality (5.6), conditions (5.2) for ;=a,+1,...,a, and (5.4) for 


JEE, we obtain 


p> c= 2 (2 a,r)x,+ (di G; ;h;) Xj. 


1eE j=n,+1 


By assumption, however, x,=0 if #€E. Hence, 


2 aishidxy= 2 di Pa, pp (5.7) 


=1 (=1 


Sex => 
C,x<,= 
me 


Further, applying the equality 


»» a,jXj;=6;, m+i<i<cm 


and condition (5.5), we have 


> x)= 2 b;A;. (5.8) 


* In Kantorovich's last book /65/ decision multipliers are called objectively-conditioned costs, This 
term follows from the economic interpretation of the solution of the dual problem. 
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According to Lemma 1.2, equality (5.8) proves the optimality of programs 
Xand A. 

Thus A=(A,,4,,...,4,) is a solution of problem (4.4)-(4. 6). 

2. Now let A=(A,, A,,---, Am) be a solution of problem (4.4)—(4.6). 

Since the vector A is a feasible program of problem (4.4)-(4.6), it satis- 
fies conditions (5.1)-(5.3). 

Let E, denote the set of subscripts of the free restraints in (4.2) and &, 
the analogous set of subscripts in (4.3). 

According to the second duality theorem, the i-th restraint in (4.6) (the 
j-th restraint in (4.5)) is fixed if i¢E (JE £,). Hence 


4,=0 for i€E; (5.9) 


nia 


a;A,=c, for /€E,. (5.10) 


Consider a solution X=(x,, x,,..., %,)0f problem (4,1)-(4.3). If x,;>0, 
then, by definition, EE, and (5.10) applies. If } a;,%,<b;, then 1€£, and 


(5.9) holds. The vector A thus satisfies eenditions (5.1)—(5.3) and is as- 
sociated by (5.4), (5.5) with some program of problem (4.1)-(4.3) (any solu- 
tion of the problem may be taken as a suitable program). Hence A=(A,, 

Ayr -++,A,) iS a decision vector of problem (4.1)-(4.3). This completes the 
proof, 

5-2. The optimality criteria of programs of linear-programming prob- 
lems are more easily stated in terms of decision multipliers. 

Theorem 5.2. (Optimality criterion of programs of problem (4.1) -(4.3).) 
A feasible program X=(x,, %,,..., %,)0f problem (4.1)-—(4-3) is optimal if and 
only if a decision vectory A=(A,, hy, .+-, 4,)exiSts, associated with this pro- 
gram by (5.4), (5.5). 

Proof, Necessity. Let X=(x,, x,,..., x,) be a solution of problem (4.1)- 
(4.3). According to the first duality theorem, the dual problem (4.4)—(4.6) 
is solvable. 

Let A=(A,, A,, ..., 4) be an optimal program of this problem. Then, ac- 
cording to Theorem 5.1, the vector A is a decision vector of problem (4,1)— 
(4.3), and, moreover, as has been shown in the proof of the second part 
of Theorem 5.1, the vector A is associated by (5.4), (5.5) with any solution 
of problem (4.1)-(4.3) and, consequently, with the solution xX under con- 
sideration. 

Sufficiency. Assume that a decision vector A=(i,, A,,..., A,)exists, 
associated with the given program X of problem (4.1)-(4.3) by (5.4), (5.5). 

In our proof of the first part of Theorem 5.1 we established (5.8). This 
equality (remembering that X and A are feasible programs of problems 
(4.1)-(4.3) and (4.4)-(4.6), respectively) indicates optimality of program X 
(see Lemma 1.2). This completes the proof. 

This criterion gives us a fairly convenient tool for deciding whether a 
given feasible program is a solution of the problem. The general scheme 
of testing programs for optimality is as follows. 

Using the system of equations (5.2), (5.4), and (5.5) determine the vector 
A=(A,, 4,,...,4,). Then, by substitution check whether this vector satisfies 
conditions (5.1), (5.3). If it does, the vector X is a solution of the problem; 
otherwise it is not an optimal program. 

The optimality criterion is generally employed when analyzing support 
programs ofa problem. We will deal with this aspect in more detail. 
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Consider a support program X of problem (4.1)-(4.3). Let 


=0 for j=1, 2,...,,<n,, (5.11) 
*1\>0 for JPN ly i254 Bee , 
a <b; for i= }, 2, eee y m= M,, (5.12) 
B o74/4 =6b; for f=m,+1,..., m,, 


Write out the matrix whose elements are coefficients of the restraints 
reduced to equalities by program X: 


1 0 eee 0 @ee 0 
. e e e e e ee 7. se @®© © © © «© «© @ @ n, 
0 0 ei A aa J 
Ax= Qmatiy1 Ont, 2 eee Ginati, nm; =e ae Qniti,n m—m. 
e e e @ e e e e . ce ee ee ge oe e 2 
Qn ans eoce Gan, oee Gan 


Since X is assumed to be a support program, there are A,linearly inde- 
pendent rows among the m+n,—m, rows of matrix a. The unit vectors con- 
stituting the first Ay rows of matrix a, are linearly independent. Of the re- 
maining m—m, rows we may, therefore, choose n—n,rows which, together 
with the first 2,rows of matrix Ay, constitute a linearly independent system. 

Write this linearly independent system as a determinant of order a and 
expand it in the first a, rows. A nonzero determinant of order n—n, whose 
rows are some of the vectors of the system 


(Qi, nytis Qi, nj+2) 2205 Qn)s i=m,+1, coe M, (5.13) 


is obtained. Hence the system (5.13) of (n—2a,)-dimensional vectors is of 
rank na—av,. 

If we assume that the program X is nondegenerate, the number of rows 
in Ay is equal to the number of columns, i.e. , 


mM—m, +n, =N. 


In this case the number of vectors, m—m,, in system (5.13) coincides with 
the rank of the system, n—a,. 

Thus, if X is a nondegenerate program, (5.13) comprises n—n, linearly 
independent (n—n,)-dimensional vectors. If, however, X is a degenerate 
program, the number of vectors in system (5.13) is greater than its rank. 

Following these preliminary remarks we now test program X for opti- 
mality. We shall try to find a decision vector A=(A,,4,,...,4,) associated 
with this program by (5.4), (5.5). Applying (5.4) and (5.11), we obtain 


Day) = Cy for j/=n,+1,2,+2,..., a. 
On the other hand, according to (5.5) and (5.12), 
A,=0 for b= 1, 2p chy 


Hence, the remaining m—m, components of the decision vector A (if it 
exists) must satisfy the system of equations 

mm 

> A@.=c, J=ntl,...,n. (5.14) 


t=, +1 


The columns of the matrix of coefficients of (5.14) are the vectors (5.13), 
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The rank of system (5.14) is, therefore, n—a,, i.e., equal to the number of 
equations in this system. 

The analysis of program X proves to be very simple especially in dealing 
with a nondegenerate program. In this case the number of unknowns in 
(5.14) is equal to the number of equations in the system and there exists a 
unique solution. Solving the system, we obtain a unique vector A which 
satisfies conditions (5.2), (5.4), and (5.5). We now substitute the vector A 
into the left-hand sides of (5.1), (5.3), which in this case have the form 


f=m,+ 


> tipi > Cp Jal, 2, cee My (5.15) 


N,22=0, i=m,+1,...,m,. 


If all the conditions (5.15) are satisfied, A is a decision vector. According 
to the optimality criterion, this proves that the program xX associated with 
A by (5.4), (5.5) solves the problem in question. 

If, however, at least one of relationships (5.15) does not hold, no deci- 
sion vector for program X exists and, consequently, this is not an optimal 
program, Observe that the last proposition is a consequence of the exist- 
ence of a unique solution of system (5.14). 

The situation is more complicated if X is a degenerate program. [In this 
case the number of unknowns in (5.14) exceeds the number of equations and 
the system has an infinite number of solutions. According to the optimality 
criterion, X is an optimal program if and only if at least one solution of 
system (5.14) satisfies conditions (5.15). 

Let us solve (5.14) for some nz—a, unknowns substituting the results into 
conditions (5.1), (5.3). We obtain a system 7 comprising 


t=m,—m,+n, 


inequalities relating 
sS=m—m,+n,—2n 


unknowns. Optimality of program X is equivalent to solvability of this sys- 
tem of inequalities. The rank of system T is, obviously, s. If the system 
of inequalities T is solvable, the set of its solutions forms a polyhedral set 
with vertices. 

Hence, to establish solvability of T it suffices to find solutions of all 
systems of equations comprising s linearly independent restraints entering 
T. The number of such systems obviously does not exceed Cj. If at least 
one of the solutions satisfies the other relationships of T, the given system 
of inequalities is solvable and, consequently, X is an optimal program. 
Otherwise the system of inequalities T has no solutions, which proves that 
X is not an optimal program. 

When s and G are quite small, the method outlined for establishing the 
solvability of system T is quite practicable. If, however, the value of 
these parameters is large the method involves many computations. Asa 
result, an improved version of the method is used in linear programming: 

(a) the transition from one system of s equations in s unknowns to 
another system is ordered, so that the analysis of all the systems becomes 
superfluous; 

(b) each transition is performed with the aid of simple recurrence for- 
mulas. 

Summing up, we reach the following conclusions: 
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1. Practical application of the optimality criterion to nondegenerate 
Support programs reduces to solving one system of linear equations. 

2. In the degenerate case, the optimality criterion involves analysis of 
a system of inequalities, which is equivalent to the solution of several sys- 
tems of linear equations. 

9-3, It follows from Theorem 5.2 that the decision vector may be re- 
lated by conditions (5.4), (5.5) only with the optimal program (4.1)-(4.3). 
On the other hand, to each optimal program X of this problem a character- 
istic decision vector related to it by (5.4), (5.5) exists. The question arises 
as to whether some relationship exists between the decision vector of any 
optimal program of problem (4.1)-(4.3) and an arbitrary solution of this 
problem. The following proposition answers this question. 


Theorem 5.3. (Dectsion-vector theorem.) The decision vector of some 
optimal program of problem (4.1)-(4.3) is also a decision vector for any 
other solution of this problem. 

Proof. Let A=(A,,A,,...,4,) be the decision vector of program X, i.e., 
a decision vector of problem (4.1)-(4.3) associated with X by (5.4), (5.5). 
According to Theorem 5.1, the vector A solves the dual problem (4.4)—(4.6). 
In the proof of the second part of Theorem 5.1, we showed that any solution 
of the dual problem is related by (5.4), (5.5) with any solution of the primal 
problem. Hence A is a decision vector for any optimal program of prob- 
lem (4.1)-(4.3). This completes the proof. 

The above theorem establishes a relationship between the concept of a 
decision vector and the set of all optimal programs of the given problem. 
We may, therefore, speak of a decision vector of a linear-programming 
problem without referring this vector to any particular program. 

We emphasize again that in general finding a decision vector of the prob- 
lem (the decision multipliers) is in no way simpler than solving the prob- 
lem. However, as stated above, finding a decision vector is equivalent to 
solving the dual problem. The principal importance of decision multipliers 
lies in the fact that they are most convenient when stating optimality cri- 
teria widely used in computational methods and theoretical applications of 
linear programming. 

In describing linear programming methods we shall generally deal with 
the canonical form of problems. It is, therefore, expedient to formulate 
an optimality criterion for this class of linear-programming problems. 

Consider problem (1.1)-—(1.3) which is obviously a particular case of 
problem (4.1)-(4.3) with m,=0,n,=n. The definition of a decision vector for 
this problem is as follows. 


A vector A=(A,,A,,...,4,) is calleda decision vector of problem 
(1.1)-(1.3) if 
(a) Di 2,4; = Cp j=; 23 Seg Ue; 
f=1 


(b) for some program X=(x,,x,,...,%,) Of problem (1.1)-(1.3) 


D2), 4; = Cy, if x, > 9. (5.16) 


Theorem 5.4. (Optimality criterion for problem (1.1)-(1.3).) <A feasible 
program xX of problem (1.1)-(1.3) ts optimal if and only tf a decision vector 


exists, related to X by (5.16). 
This theorem is a particular case of the general optimality criterion 
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(Theorem 5.2), In practical applications the remarks given in 5-2 should 
be taken into consideration. 

5-4. The method of Lagrange multipliers is usually used in solving con- 
ditional extremum problems in classical analysis. We review briefly the 
essential points of this method. 

Suppose it is necessary to maximize or minimize a function 


FUSS FP (SX ees Sah (5.17) 


whose variables are related by the conditions 


G; (X) = G; (X,, %, ---, X,) =9, 
i=1,2,...,m (m<n). (5.18) 
Let the functions F(X) and G,(X), i=1, 2, ..., m, be continuous and have con- 


tinuous partial derivatives of the first order with respect to all their 
variables. 

We say that the system (5.18) is regular at point X=(x,,x,, ...,x,) 
if the determinant of the matrix 


0G, (X) 9G, (X) 0G, (X) 
Ox;, Ox;, Ox; 
0G,(X) 9G, (X) dG, (X) 

Ox;, Ox;, Ox; 


e e© © ss 8 © 8s ss *# ee @ # 8 @ © 


does not vanish. Here i,,i,,...,/, are any m subscripts of (1, 2,..., 2). 
Lagrange's method is based on the following: 
If a function (5.17) subject to conditions (5.18) attains a maximum or a 
minimum at point xX and the system (5.18) is regular at this point, there 
exist numbers i,,),,..->h_ Such that the function 


Fx (X)= F(X) + 2G;(X) (5.19) 


satisfies, at point X, the necessary conditions of unconditional extremum, 
22% 


OF 4(X) 


ax, =0 for i=1,2,...,a. 


A, Ay, ---,4, are generally termed Lagrange multipliers, and the 
function Fa(X)— Lagrange function. 

Computation of the conditional extremum (5.17), (5.18) thus reduces to 
determining the unconditional extremum of the Lagrange function (5.19). 
The Lagrange function with the unknowns 4, is determined. Then, the sys- 
tem of equations 


OF , (X 
eu 6. i=1,2,...,” 
x; 


is solved. The solution of this system depends on the unknown parameters 
\;, d=1,2,..., m, which are determined from (5.18). 

The mathematical-programming problem differs from the classical prob- 
lem of conditional extremum in that inequality restraints appear. Lagrange's 
method is, therefore, inapplicable in this case. However, after some 
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modifications the method can be extended to a fairly general class of mathe- 
matical-programming problems, too. Here we consider only the case of 
linear programming. 

Let the linear-programming problem (1.1)-(1.3) be given in canonical 
form. Let 

Aa 
F(X)= Beye’ 
G(X) = — Bi aj%)+p i= 1, 2, coo, MM 


Theorem 5.5. A feasible program X of problem (1.1)-(1.3) is optimal 
if and only if the Lagrange function 


Fa (X)= F(X) + B10; 


for some values of the multipliers 4,(i=1, 2, ..., m) attains, at point X, a 
maximum subject to the condition 


x, 0, i=1, 2, eeey R. 


Proof. Necessity. Let X=(x,, x,,...,%x,) be an optimal program of prob- 
lem (1.1)-(1.3). Let the vector A=(A,,4,, ....4,) be an arbitrary decision 
vector of the problem. Let 


Fa(X)= Dew + a (— Baye, +0,) = Bx,(4— Bray) + Be 


From the definition of the decision vector, 


c,— Yrja;,<0, J=1,2, ...,4% (5.20) 
i=1 
where 
ey— ha, =0, (5.21) 


if x,>0-. Applying (5.21), we rewrite the expression for F,(X) in the form 
m m 
Fa(X)—= 3 2j(e— Bhai) + Dido 
where E is the set of subscripts j for which x, == 0. Therefore, 
A. =e m m 
Fx (X) — Fa (X) = Pr (x,—%,) (¢, =e aijh;) = — 24) (4-3 a, hi e 


By assumption, x,>0 and the multipliers 4, satisfy inequalities (5.20). 


Hence, ae 
Fa (X) — Fa(X) 20 


for any vectors X with nonnegative components. 
Sufficiency. Leta feasible program X of problem (1.1)-(1.3) satisfy the 
condition 


— 


Fx (X) = max F(X) (5.22) 


for some A=(A,, A,, -+-1A,_) Consider 


If for some j m 
c;— Da; jhi >9, 
(=1 


then, letting the component x, of vector X increase to infinity we obtain 


sup F(X) = oo, 
X po 


which contradicts (5.22). Hence 
4 — Boyd, <0 for j=1,2,..., 4”. (5.23) 
Analogous considerations show that 
y—Layi=0 for %,>0. (5.24) 


Indeed, if this relationship does not hold, then somewhat modifying the /-th 
component of the vector X we may form a vector X such that 


F(X) > Fa(X), 
which contradicts (5.22). 

According to Theorem 5.4, relationships (5.23) and (5.24) prove the op- 
timality of program X. The vector A appearing in the function F, is a 
decision vector of the problem. This completes the proof. 

The theorem proved above enables us to refer to the components of the 
vector A=(A,,4,,...-4,) appearing in the definition of the function Fy, for 
problem (1.1)-(1.3) asthe Lagrange multipliers of this problem. In 
the proof of Theorem 5.5 we established, in particular, that the set of all 
decision vectors of the linear-programming problem coincides with the 
system of vectors constructed on the Lagrange multipliers of the problem. 
Decision multipliers and Lagrange multipliers of a linear-programming 
problem are thus equivalent concepts. 

In our formulation of Theorem 5.5 the vector X was assumed to be a 
feasible program of the problem in question. This theorem, therefore, 
does not entirely eliminate the necessity of taking into consideration re- 
straints (1.2) relating the variables of problem (1.1)-(1.3). In order to eli- 
minate (1.2), we must find a saddle point of the Lagrange function. 

We now give the necessary definitions. 

Let R(X, Y) be a function of a vector X, which delongs to the set Ty, anda 
vector Y, of the set Ty. 

A point (X,, Y,) € 7, x T,* is called a saddle point of the function R(X, ¥Y) 
subject to the condition (X, Y)E Ty x Ty, if 


R(X, Y,)< R(X, Y,) << R(X,, ¥) (5.25) 


for all the points (X, Y)ET, x Ty. Inequalities (5.25) show that the maximum 
value of the function R(X, Y,) on the set Ty is obtained at the point X,, and the 
minimum value of the function R(X,, Y) on the set 7, at the point Y,. 

We now consider the linear-programming problem (4.1)—(4.3). Restraints 
(4.2) comprise equalities and inequalities. The nonnegativity requirement 
is imposed only on part of the variables. Let as before 


n 


Dix (2p. 


F, (X)= F(X, A)= p> C;X, + 2 Ni (6, = 2 a,x; ) = 2 Cx, + % 4,6;— > 


(=1 


* By definition V=(u, v)ETy x Ty, if uET x, vETy. 
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Theorem 5.6. The vectors X*=(x*, x, ..., x*)and A*=(a, AS, ..., M)are, 
respectively, a solution of problem (4.1)-(4.3) and its decision vector if 
and only if (X*, A*) ts a saddle point of the function F(X, A) subject to the 
conditions 


x, 20, jJ=1, 2, 4 tage llys 
A;==0, i=1, 2,...,m (5.26) 


1° 


Proof. The proof is based on arguments very similar to those used in 
the proof of the Theorem 5.5. The suitable changes are left to the reader 
(see Exercise 13). 

According to Theorem 5.6, a pair of conjugate problems (4.1)-(4.3) and 
(4.4)-—(4.6) is equivalent to the problem of finding a saddle point of the La- 
grange function F(X, A) subject to the conditions (5.26). 

Theorem 5.6, formulated here for linear-programming problems, can 
be extended to a wide class of nonlinear-programming problems. 

This generalization of Theorem 5.6 provides a theoretical basis for some 
numerical methods of nonlinear programming. 

5-5. To conclude this section, we give another interpretation of decision 
multipliers of linear-programming problems. We limit the discussion to 
problem (1.1)-(1.3) written in canonical form. 

Let A=(A,, A,, ..., A,) be a decision vector of problem (1.1)—(1.3) or, 
equivalently, a solution of the dual problem (1.4)-(1.5). We shall now show 
that the components A, of vector A may be interpreted as the influence of 
the various restraints of system (1.2) on the magnitude of the maximum in 
problem (1.1)-—(1.3). We state this proposition more precisely. 


Theorem 5.7. Let problem (1.1)-(1.3) be nondegenerate and let M(b,, 


b,, ..., 5,)be the maximum of its linear form under conditions (1.2), (1.3). 
Then 
Dye Oe ns Pa), J=1, 2, 025 m. (5.27) 


Proof. We denote problem (1,1)-(1.3) by (Ag). Let X*=(x', x§,...,*«°) bea 
support solution of (A,;). Without loss of generality we may take the first 
components of the vector m to be nonzero. The restraint vectors A, /=1, 
2, ..., m are then linearly independent and any m-dimensional vector B’=(6, 


b’, ..., 6°) can be written as a linear combination of the restraint vectors 


B= 3 Ap (5.28) 
<1 
Let |je,||, denote the inverse of the regular matrix (A,,A,,....4,). Then, from 
(5.28), we have 
> e,0,, tal, 2 0. me (5,29) 
1 
Let 
m 
= ei; x=minx’. 5.30) 
% ee | u i<i<m ‘ ( 


Consider a linear-programming problem (Azg’) which is obtained from prob- 
lem (A,) by replacing B by 8. We shall show that when 


max |b; —b; |<< = (5.31) 


ici<m 
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the n-dimensional vector X’=(x), x), ..+, ¥,, 0, 0, ..., 0), where the components 
x, are defined in (5.29), is a solution of problem (Ag)). | 

We shall first show that X’ is a feasible program of problem (Ag). Ac- 
cording to (5.29), 


Li 
x,—x=)) €,,(0,—5,), {= 1, 2, coey MM. 
J=1 


Hence, applying notations (5.30) and condition (5.31), 
[x,—x; |< emax|b),—6,| <x, 
and, therefore, 
x, xi —x S0. 


The vector X’ thus satisfies restraints (1.3) of (Ag). The restraints (1.2), 
with B’ substituted for B, are satisfied by xX’ because of the particular de- 
finition of the components x’, x{, ..., x, (see (5.28)). xX’ is thus a feasible 
program of (Ag). 

Let A=(A,, Ay, +--+, 4) be an optimal program of problem (1.4)-(1.5) (a de- 
cision vector of (A,)). According to the optimality criterion of programs 
of (Az) (see Theorem 5.4), we have 


m 


Sor | ie se Sige es tig (5,32) 
at 2c, j=m+l, m+2, ..., A. 
Relationships (5.32) show that A is also a decision vector of Ag related by 
(5.16) with the feasible program xX’. Vector xX’ is thus a solution of Ag. 
When conditions (5.31) are satisfied, X’ is a solution of Ag, and A (in- 
dependent of the choice of 8’) is a solution of the dual problem with respect 
to Ag:. 
According to the first duality theorem 


m 
M(b,, b-.-6 OY =P Oy (5.33) 
f=1 
where 8’ is restrained only by (5.31). 
Since 
x= min x® 
igiam 


is positive (the program X* is, by assumption, nondegenerate), we conclude 
that (5.33) holds for any vector B’ lying in some given neighborhood of 
vector B. The required equalities (5.27) can now be obtained by differentia - 
tion of (5.33). This completes the proof. 

Thus, in the nondegenerate case the components of a decision vector of 
problem (1.1)-(1.3) give an appraisal of the influence of the right-hand sides 
of restraints (1.2) on the magnitude of the attainable maximum of linear 
form (1.1). 

In the proof of Theorem 5.7 we have not fully drawn on the assertion that 
all the support programs of a problem are nondegenerate. It was only 
assumed that one of the optimal programs is nondegenerate, It is noteworthy 
that the last requirement is essential. If this condition is not fulfilled, the 
solution of the dual problem is generally not unique (see 6-2). Formula 
(5.27) will, therefore, not hold in general: the function M(b,, ..., 6,) need not 
have partial derivatives. However, in the degenerate case, too, our inter- 
pretation of the decision multipliers remains valid (though in a somewhat 
modified form). 
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Formula (5.27) was derived for the linear-programming problem in ca- 
nonical form. Obviously, however, this formula applies to any problem 
with a nondegenerate support solution. We suggest that the reader prove, 
as an exercise, Theorem 5.7 for problem (4.1)-(4.3) (see Exercise 14), 


§ 6. Some applications of the duality principle 


The duality of conjugate linear-programming problems established in this chapter is most useful in linear 
programming and in related mathematical disciplines. The duality principle is used in the construction of 
various numerical methods of linear programming. This aspect is dealt with in Chapters 6 and 7, Moreover, 
the concept of duality is widely applied in qualitative investigations of various mathematical problems, The 
principle of duality becomes particularly effective when extended to infinite-dimensional spaces. However, 
even in the finite-dimensional case, which is considered here, duality simplifies the analysis of certain mathe- 
matical problems. We shall illustrate this point by several examples, 

6-1. Consider the linear-programming problem of maximizing the linear form 


m 


> CHI (6. 1) 


: on é=1 
subject to the conditions 


m 
D4iyiaed, Jol, % oa. (6.2) 
é=1 


We shall assume that problem (6.1)~(6.2) is solvable. In this case, all its solutions (the optimal set of 
the problem) is a convex polyhedral set M*. We shall determine the dimensionality of the optimal set M* 
and, in particular, establish the conditions under which this set is zero-dimensional (condition under which a 
unique solution of the problem exists). To this end we formulate the dual problem of problem (6.1)=(6.2). 
Minimize the linear form 


g 
dey (6.9) 
subject to the conditions ie 
x 
> dj jx j= C1, i=1, 2, ..., m; (6.4) 
i=1 
xj=0, feed 2, aan, (6.5) 


We call the restraint vector D;=(d,,, dy, ...,dmy)7 Of problem (6.3)-(6.5) free if thej-th restraint in 
(6.5) is free (see 3-4), 


Theorem 6.1. The optimal set M* of problem (6.1)-(6.2) is 0*-dimensional, where 
o* =m—,r, (6.6) 
where r is the rank of the matrix of the free restraint vectors of problem (6.3)-(6.5). 


Proof. Let E be the set of indices of the free vectors of problem (6.3)-(6.5). In this case, according to 
the second duality theorem, the j-th restraint in (6.2) for fEE is fixed, i.e., for any vector YEM* 


m 
» 4ipi=4y 
4=2 


Consequently, the optimal set M* is contained in the polyhedral set M’ defined by 


Sa; —f <dy (EE, f=1,2 0... 75 (6.7) 

| dy IEE, 11% 00, (6.8) 

We shall show that any vector Y of M’ is a solution of problem (6.1)-(6.2), i.e., belongsto M* 

Let X*=(x,, x, or x) be some solution of problem (6.3)—(6.5). From the definition of the set E, 
x; =0 for jE. (6. 9) 
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Consider an arbitrary vector YEM’. The following equalities hold: 


» Ci¥i => ( 2 4s sj) = 


f=1 é=1 


The first equality follows from (6.9) and (6.4), the third and fourth follow from (6.8) and (6.9), respec- 
tively. The second equality holds when the order of summation is reversed, 
Thus, 
Yai Dee, (6,10) 


i=l 


According to Lemma 1.2, (6.10) indicates that program Y is a solution of problem (6.1)-(6.2). 

M* thus coincides with M’ and is, consequently, defined by conditions (6.7), (6.8). 

From the definition of E, any j-th restraint of system (6.5) when i€E is fixed. Hence, according to 
the second duality theorem, the j-th restraint in (6.7) (j¢£) is free. In other words, for any jGE there 
exists a vector YY) E€M® so that 


2 ay 4s <4 


Ma 


All the conditions of system (6.7) thus impose nonrigid constraints on the polyhedral set M* (see Chapter 2, 
1-2), 

It now remains to apply Theorem 1.2, Chapter 2, and (6.6) follows. This completes the proof. 

As a corollary of Theorem 6.1, we derive the necessary and sufficient conditions for the existence of a 
unique solution of problem (6.1)—(6.2). 

Theorem 6.2. The solution of problem (6.1)-(6.2) is unique tf and only if there are 
linearly independent restraint vectors among those of problem (6.3)-(6.5). 

The proof follows immediately from (6.6) for r=m. 

We now give one sufficient condition for the existence of a unique solution of problem (6.1)—(6. 3). 

Theorem 6.3. Uf at least one of the optimal support programs of problem (6.3)-(6.5) is 
nondegenerate, problem (6.1)-(6.2) has a unique solution. 

The proof follows directly from the sufficiency of the conditions of Theorem 6.2. 

If we assume that problem (6.3)—(6.5) has a unique solution, the conditions of Theorem 6,3 prove to be 
not only sufficient but also necessary. Indeed, let X*=(x,,2 Bc x.) be the unique solution of problem 
(6.3)=(6.5). In this case the restraint vector D, is free if and: only ‘if x;>0. If the support program X* is 
degenerate, the number of x;>0 is lessthan m, Hence, from Theorem 6,1, the dimensionality of the 
optimal set M* is greater than zero, which means that (6.1)~(6.2) has not a unique solution. 

If problem (6.3)—(6.5) has many solutions, the conditions of Theorem 6.3, generally speaking, are only 
sufficient. This can be verified by a suitable example, the construction of which is left to the reader 
(Exercise 15). 

6-2. Until now we studied problem (6.1)—(6.2) with fixed parameters c;, d;; and d,. We now slightly 
modify the statement of our object and consider all solvable problems (6.1)-(6.2) with equal ¢;,4;; and 
arbitrary dy The set of linear-programming problems which is obtained in this way wil! be denoted by 
H (c;,d;/). We are interested in finding the necessary and sufficient conditions for a unique solution of all 
problems of this class. 


Theorem 6.4. Nondegeneracy of problem (6.3)-(6.5) is a necessary and sufficient condi- 
tion for each of the problems of H (c;, d;;) to have a unique solution. 


Proof. Sufficiency is an obvious corollary of Theorem 6.3. We will prove necessity. 
Let problem (6.3)-(6.5) be degenerate. This means that it has at least one degenerate support program 


X* a(x", x, ..., xn) 
To be specific, let n 


o{ >0, j=1, 2, eros FM, 
*} 
=0, j=r+l, 
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Let Y=(¥,, Ygs «+++ Yq) be a solution of the system of equations 


m 
Ddiyi=dq, 1=1,% ... 7, (6.11) 
é=1 
where dy, j=l, 2, ..., r are arbitrary numbers. For dy; when j>r we take any dy satisfying the condition 
m — 
Di dipi<dp  ferth, r42 0,0 (6.12) 


(=1 


We shall show that a problem of H (c;, dij) for d yuty has several solutions. 

Let YO —=(y), yl, ©... y)40 be a solution of the homogeneous system of equations corresponding to 
system (6.11). Existence of this vector follows from the fact that r<m. We form a system G comprising the 
vectors Y,=Y-+eY), where the parameter e is chosen so that the vector Y, is a feasible program of prob- 
lem (6.1)=(6.2). Obviously, there exists a number @>0 suchthat Y, for je|<a satisfies (6.2). The sys- 
tem G therefore comprises an infinity of vectors, each of which is a solution of system (6.11). Let Y be any 
vectorof G. Following the reasoning used in the proof of (6.10), we obtain 


™ a = 
D1 y¥i=D dy «}. 
fst f=1 
The vector Y is thus a solution of problem (6.1)=(6.2). 

Assuming degeneracy of problem (6.3)-(6.5) we have thus found a problem of H (c;, dj;,) with several 
sOlutions. Necessity of the conditions of Theorem 6.4 is thus established. 

Let 


Pee ae 
AG, Ja aces la-v= 2f 2f 2/m-1 a s 


As a corollary of Theorem 6.4, we shall establish a sufficient condition for problems of 4H (¢;, djs) to have 
a unique solution. 

Theorem 6.5. If for any system of linearly independent restraint vectors Dj, Dy, -++: D;,-, 
of problem (6.3)-(6.5) the determinant A(j,, jg: ..-1 im-1) does not vanish and the matrix (D,, 
Da. ., Dn) is of rank m, all the problems of H (c;, d;) have unique solutions. 

Proof. Let the system of vectors Ds,, Ds, --+, Ds, constitute the basis of some support program X= 
= (x,,%, . ., X,) of problem (6.3)-(6.5). In this case x;=O for jf#s_, k=1, 2, ..., m. The components 
of program X expressed in term of the basis vectors can be determined from the system of equations 


m 
dis, Xs, =Ch, é=], 2, ..., m. 
2 k °R 
Hence, applying Cramer's rule, we obtain 


X,, (= 1)-"4 (Si, Se, +s Show Seti +++» Sm) k 


where A is the determinant constructed from the components of the vectors Dg,, Dg, «+s Ds. By assump- 
tion, however, 


A(s,, S83, rr | Spe—y Sp+is one, Sm)#0. 
Therefore Xp, #0 for k=1, 2, ..., m. The vector X is thus a nondegenerate program of the problem. 
Since X is an arbitrary support program of problem (6.3)=(6.5), this problem is nondegenerate. Hence, 
from Theorem 6.4, each problem of A (cp dij) has a unique solution. This completes the proof. 


6-8. To give another application of the duality principle, we shall prove two basic theorems of the theory 
of linear inequalities. Consider a system of linear ineaualities 


m 
Dawid, 1=1, 2 0... 0. (6.13) 
{=1 
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Theorem 6.6. (Consistency of system (6.13).) The system of inequalities (6.13) ts con- 
sistent if and only if any nonnegative solution X =(x,, Xy «++, Xn) Of the homogeneous system 
of equations 


n 
Dd 4ixs=0, i=], 2, ...,m, (6.14) 
f=! 
satisfies the condition 
n 
» 4/20. (6.15) 
ji 


Proof. Necessity is easily established. Indeed, let system (6 13) be consistent and let Y=(y,, Yg, ---» Ym) 
be one of its solutions Multiplying the j-th inequality in (6.13) by the nonnegative number x, and summing 
the products, we obtain 

m 
z Yi (3 #1) < >> gel (6.16) 
If the vector X=(x,, X, ..., X,) satisfies equations (6.14), inequality (6 15) follows from (6.16) 

We now prove sufficiency. Consider the linear-programming problem (6.1)-(6.2), where co; =0 for i=], 
2, ..., m. We shall denote this problem by (A). Any solution of system (6.13) is obviously a solution of 
problem (A) and, conversely, any solution of problem (A) satisfies (6.13). The consistency of system (6.13) 
is therefore equivalent to solvability of problem (A). 

Let (A) be the dual problem with respect to (A). Problem (A) has the form (6.3)-(6.5) force;=0, i=1, 

2, ..., m. Let the requirements of the thorem be satisfied. In terms of problem (A) this means that the 
linear form of problem (A) is bounded below by zero in the set of its feasible programs. Since the set of 
feasible programs of problem (A) is a priori nonempty (one of its programs is the zero vector), problem (A) 
is solvable according to Chapter 2, Theorem 4,4, In this case, however, the first duality theorem guarantees 
the solvability of problem (A), which is equivalent to consistency of the system of inequalities (6.13). This 
completes the proof. 

Note that this theorem is a natural analog of the well-known consistency criterion for systems of linear 
equations (see Appendix, Theorem 2.6), which is stated as follows: 

A system of linear equations 


m 
Daiyizd, j=l, 2% ..., 0, 
{=1 


is consistent if and only if any solution 


A (Xi cd x) 
of the conjugate homogeneous system 


n 
D 4ijx;=0, i=l, 2, ’ m, 
j=1 
satisfies the relationship 
n 
Ds d jx ;=0 
f=1 
Together with system of inequalities (6.13), consider an arbitrary inequality 
m 
> Ci; KC. (6.17) 
= 


A question arises: under what conditions does inequality (6.17) follow from (6.13)? 
The answer is provided by the following theorem. 


Theorem 6.7. Inequality (6.17) follows from the consistent system (6.13) if and only if 
there exist nonnegative numbers x, x, ease z such that 


SD d2=4 i=l], 2, eer, m, (6.18) 
j=) 
n 
d “ <. 
» wy (6.19) 


2154 


118 


Proof. Sufficiency is almost obvious. Indeed, let the requirements of the theorem be fulfilled and let 
Y =(9,, Yar «s) Ym) DE a solution of system (6.13). Multiplying the j-th inequality in (6.13) by xj and 
summing the products, we obtain, applying (6.18), 


m n ‘ 
Ds ii SD) a j*}. 
é=1 j= 


Further applying inequality (6.19), we conclude that Y satisfies (6.17). 

To prove necessity, consider the linear-programming problem (6.1)-(6.2). The value of the linear form 
(6.1) of any feasible program of problem (6.1)—(6.2) is at most ¢ (any solution of system (6.13) satisfies (6.17)). 
The set of feasible programs of the given problem is nonempty (inequalities (6.13) are consistent). There- 
fore, according to Chapter 2, Theorem 4.4, problem (6,1)—(6.2) is solvable. 


Let Y* =(y\, Ys» er y,,) be a solution of problem (6.1)—(6.2). Since the vector Y*satisfies inequalities 
(6.13), it. of 
Lewis. (6.20) 
=1 


According to the first duality theorem problem (6.3)-(6.5), dual with respect to thesolvable problem (6.1)= 
(6.2), has a solution X* = (x1, ty err x7) which satisfies the equality 


m n 
Did = 4%}. (6.21) 
é=1 j=t 


The vector X*is a feasible program of problem (6.3)=(6.5), Its components, therefore, satisfy conditions 
(6.18). As regards inequality (6.19), it is satisfied by virtue of (6.20) and(6.21). This proves necessity. 

Like the previous theorem, this proposition is also an analog of a corresponding assertion for linear 
equations: 

The equation 


m 
>» ot eed 


{=1 


follows from the system of equations 
m 
Da4iyimdp Fl, 2, 042, 
i=1 


if and only if there exists x}, x,,...,x,, Such that 


n 
D4 =e b= 1, 2543 
j=1 


n 
> x,dj=0, 
Jat 


EXERCISES TO CHAPTER 3 


1, Formulate the dua] problem with respect to the transportation problem (see Chapter 1, 3-2). 

2. Formulate the dual problem with respect to the problem of national-economy planning discussed in 
Chapter 1, 7-3. Give the economic interpretation of the dual problem. 

3. Let the vectors T; #0, i==1, 2, ..., N; and let Tp be a convex polyhedral cone spanned by the 
vectors 7, and having its apex at . For apex P to be the point of cone Tpit is necessary and sufficient that; 

For any y;>0, 


N 
D> viT;=20 
fe. 
entails 
9, =¥s=-0 =yv=0 
Prove. 


4, Prove that the set 7%) introduced in the proof of Lemma 2.3 is a convex polyhedron (see Chapter 2, 
Exercise 13). 
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5. Prove that Lemma 2.3 applies if Tpis an arbitrary convex cone with its point at P. 

6. Give examples showing that if the assumptions of Lemma 2,3 concerning the cone K or the point P 
are changed, the lemma, generally speaking, does not hold. 

7%. Show that there does not exist a pair of dual linear-programming problems whose linear forms are 
unbounded in the corresponding sets of feasible programs. 

8. Show that the problem with homogeneous restraints (1.17)=(1.19) is solvable if the components of its 
constraint vector are nonnegative and one of the following conditions is satisfied: 

(a) for someé, all aj,>0. 

(b) a;70 for all ¢ and j and for any j there exists at least one positive jy. 

9, Prove-the first duality theorem for maximization of the linear form 


An 
L (x) = Dery (2) 
sl 
subject to the conditions 
L | 
G; X= b1, é=1,2,..., N, 
> iP y (2) 


by showing that if X* is a solution of problem (1)-(2), the point 
C=(C,, Cay oo ey Cy, L(X*)) 
belongs to the convex polyhedral cone K spanned by the vectors 


A? olan: Gis ois Cs by Peel Bp cca WV, 
and 
ent, =(0, 0, ..., 0, 1). 


Hint: Prove the last proposition by "reductio ad absurdum”, applying Corollary 3.2, Chapter 2, to 
point C and to cone K. 

10. Verify that problem (4.1)—(4.3) is dual with respect to problem (4.4)—(4.6) (see 4-1). 

11, Use Theorem 5.2 to derive an optimality criterion for programs of problems indicated in Exercise 1. 

12, Test, for optimality, the programs (1, 0, 0, 0) and (0, 0, 1, 1) of the problem of maximization of 
the linear form 


%,—dX,+ 4X, 
subject to the conditions 
¥,—*,+2x,=I, 
2x,+%,+4x,—2x,=2, 
X%,2>0, x,>0, x%,5>0, x,>0. 


13, Prove Theorem 5.6. 

14, Prove the validity of (5.27) for problem (4.1)—(4.3) with at least one nondegenerate support solution. 

15, Show that there exists a unique solution of problem (6.1)~(6. 2) although its dual problem has no 
nondegenerate support solutions. 
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Chapter 4 
THEORETICAL PRINCIPLES OF THE SIMPLEX METHOD 


The simplex method is the method most widely used in applications 
of linear programming. The foundations of this method were laid by 
Dantzig in 1949 /40, 41/. In Soviet literature on linear programming the 
Dantzig method is sometimes referred to as the method of succes- 
sive program improvement. The term simplex method arose from 
the geometrical interpretation of the first particular problems to which it 
was applied* and, although universally accepted, it does not adequately 
describe the essential features of the procedure, 

Conceptually, the method entails three main points. First, a procedure 
for computing a support program is indicated. Second, a criterion is 
established for testing the optimality of the program selected (or, in other 
words, for deciding whether the program selected is a solution). Third, 

a procedure is proposed for transforming a nonoptimal program selected 

to another support program which is closer to the optimum. It is proved 
that the procedure yields an optimal program, i.e., a solution of the linear- 
programming problem, after a finite number of steps. Thus, the method 

is a successive improvement of program, which is expressed in the alter- 
native name, mentioned above, given to the method. 

It is noteworthy that the algorithms of the simplex method also make it 
possible to establish, during computations, whether the linear-program - 
ming problem in question is solvable. In other words, when computing it 
is possible to establish whether the problem restraints are contradictory 
and whether they ensure boundedness of the linear form. 

In this chapter we discuss only the theoretical principles of the method. 
The computational procedures are dealt with in the next chapter. 

The simplex method is presented as follows. First we give two optimal- 
ity tests for a support program of the linear-programming problem, 
Then we discuss the method in reference to the so-called nondegenerate 
case. Some support program of the problem in question is assumed to be 
given. The formal exposition is supplemented by two geometrical inter- 
pretations. 

In § 5 the simplex method is generalized and discussed for linear-pro- 
gramming problems with upper, as well as lower, bounds. Thereafter the 
limitation of nondegeneracy is dropped. 


* Restraints of the form 


tym x20, j=1, 2,....0 
=1 


define a simplex in the n-dimensional space, These restraints comprised the conditions for one of the 
first linear-programming problems for which Dantzig developed a computational procedure. 
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In §7 we describe the procedure for constructing an initial support pro- 
gram and the so-called M-method, At the end of the chapter we give some 
theoretical applications of the simplex method. 


§ 1. Optimality tests for support programs 


1-1, In Chapter 3 we derived an optimality criterion, i.e., a necessary 
and sufficient condition for a feasible program to be a solution of the linear- 
programming problem. In the simplex method we do not deal with any 
feasible program, but only with support programs. Thus, in this section 
we shall formulate an optimality test for a support program. The optimality 
test can be expressed in terms of the various characteristic parameters of 
the problem. We shall deal with two different optimality tests. Each of 
these tests will be used in one of the solution algorithms of the linear- 
programming problem according to the simplex method. Note that we speak 
here of optimality tests (rather than criteria) to emphasize that the 
considerations apply in particular to support programs and that the condi- 
tions derived are, generally speaking, not sufficient for any feasible pro- 
gram. 

Let us write the linear-programming problem in canonical form. 

Maximize the linear form 


i=1 
subject to the conditions 
DA x, =B; (1.2) 
{=1 
x,20, j=l, 2, ..., 4. (1.3) 


The problem can also be written more compactly: Compute a nonnegative 
vector (row vector) X>0 maximizing L(X)=CxX™ subject to the conditions 
AX? = B. 


Here A=(A,, ..., A,)=|[2;/lnn is the restraint matrix, A, and B being, 
respectively, the restraint vector and the constraint vector of the problem: 
A j= (Ay p Oyj, - 0) Og)”, J=1, 2, ...,% B=lb,, 6, ..., bn)7) C=(c,, Cy .-.,¢,) is the given 


coefficient vector of the linear form (1.1). We shall assume that the matrix 
A is of rank m. 

We restate the definition of a support program and its basis, these being 
the principal concepts of the present chapter. A feasible program X=(x,, 

Xx, --.,%,) of the linear-programming problem (given in canonical form (1.1)— 
(1.3)) is calleda support program if the system of the restraint vec- 
tors A, corresponding to its positive components (x,>0) is linearly inde- 
pendent. The basis of the support program is the system of m linearly 
independent restraint vectors which contains all the A, corresponding to 
the positive components of the support program. Basis components of 

the support program are the components corresponding to its basis vector; 
the remainder are called extrabasis variables. 

Let some support program X with the basis 4,, Ags: sans Aga Seed: Ape be 
given, In the following we shall find it convenient to describe any basis 
vector not only by the index s; but also by its position i in the particular 
basis (for any basis, obviously, i=1, 2, ..., m). 
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Let us take some restraint vector A, and express it in terms of the basis 
vectors A,, ... A 


° 
Sm° 


A,= DAxip J=1, 2, coos A. (1.4) 


Here x,, is the coefficient in the expansion of the vector A, in terms of the 
basis vectors corresponding to the vector A,, which occupies the /-th position 
in the basis. 

Consider the set of indices /, of the basis vectors of the support program 
X (ly is the set of subscripts of the basis variables). Obviously 


} » 
B= x,A,= XA 
z 9 eT OS 
(the extrabasis variables are all zero). The constraint vector B as expres- 
sed in terms of basis vectors is 


m 
B= DA, Xie 


where Xig = Xs, is the basis variable corresponding to the vector A, which 
occupies the i-th position in the basis. 
Denoting the constraint vector B by A,, we obtain a general formula 


Ay= DAs xp /=0, 1, 2, coy M, (1.4') 


defining the coefficients of the expansion of all the restraint vectors and of 
the constraint vector in terms of basis vectors. 

Consider the sets of the parameters z, and A,(j=1, 2, ..., a) defined as 
follows: 


£,= 2, %ip JS 1,2). ie (1.5) 
A ,=2;—Cp jJ= 1, 2, eee) RK. (1.6) 


The parameters z, and A, are determined by the support program X cor- 
responding to the basis 4,, ..., A,. In order to emphasize this we should 
write z\*) and AIX). However, to avoid unnecessary complication of notations, 
we shall omit the superscript X whenever no confusion can arise, 

The parameters A, are most significant in the simplex method: the signs 
of these parameters indicate the optimality of the program chosen. Wehave: 
Optimality test. A support program xX* is a Solution of problem 

(1.1)-(1.3) tf 4,20 for j=1, 2,..., 2. 

Proof. Consider a feasible program X=(x,, ..., x,)4X*. The components 
of the vector X, like those of any feasible program, should satisfy the re- 
straints of the problem, i.e., 


n 
B=),x,A, 
x, > 0, j/=1t, 2, aoee n. 


Expressing the vector A,, according to (1.4), in terms of the basis vectors 
of the support program X®* we obtain 
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or, reversing the order of summation, 


B= .) Ay. 1.7 
i=t (21% ;%:,) Ss; ( ) 

On the other hand, X* being a feasible program of problem (1.1)-(1.3), 
we have 


B=D x A.. (1.8) 


f=1 ‘ 


Since A,, A,, ..., A, are linearly independent, we can express the con- 
straint vector B as a unique linear combination of the basis vectors. Com- 
paring (1.7) with (1.8), we see that 


2a j%is = Fe (1.9) 


The validity of our proposition now follows from the following equalities 
and inequalities: 


~, 
tt 
i 
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Ma iMs 


. 
i] 
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The first and the last equalities give the value of the linear form L on pro- 
grams X and X*, respectively. The inequality follows from the conditions 
A,=2z,—c,20. The third and fifth equalities follow from (1.5) and (1.9), 
respectively, and the fourth equality is obtained by reversing the order of 
summation, 

Thus, if A4,>0 for all /, the support program X determines the maximum 
value of the linear form L(X) and as such solves the linear-programming 
problem. We emphasize that this condition (A4,>0 for all /) is, in general, 
only a sufficient condition for the optimality of a support program (see 
Exercise 1). 

As we have already noted, A, can be expressed in terms of the various 
parameters characteristic of the linear programming problem. Depending 
on the definition of Z, in (1.6), we have different optimality tests. 

We shall say that the optimality test is given in the first form if z, are 
computed from (1.5), Thusthe first form of the optimality 
test for support programs is formulated as follows: 

Optimality test. A support program x* is a solution of problem 
(1.1)-(1.3) if 


m 
2 Cs, Xj, = Cc; 
=1 


for all j(ij=1, 2, ..., 2). 
1-2, We now give an alternative form of the optimality test for support 


programs. 
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Let, as before, the vectors A,,, A,, ..., 4,, Constitute a basis of the sup- 
port program X*. We determine the vector A=(A,, A,, ..., 4,) from the sys- 
tem of equations 
m 
Pa aihi=ep — fElx. (1.10) 

The second form of the optimality test for support 
programs can be stated as follows: 


Optimality test. A support program x* of problem (1.1)-(1.3) is 
optimal if 


m 
aii ee (1.11) 
for J=\, 2, ..., 2. 
Proof. This sufficient condition for optimality of a support program is 
easily proved on the basis of the optimality criterion established in Chap- 


ter 3 (see Theorem 5.4), The above test also follows from the following 
equalities: 


m m m m m m 


The first equality coincides with definition (1.5) of z,. The second equality 
follows from (1.10). The third is obtained hy reversing the order of sum- 
mation. The fourth follows from (1.4) written for the components of the 
restraint vectors. 

We now see that the relationship 


Mm 
p> A; Ay = C, 
for all / is equivalent to the condition 


A, 20 for jJ=1, 2, ooey AL 


We have thus proved not only the applicability of the second form of the 
optimality test, but also the equivalence of the two forms. 

As we have already indicated, to each form of the optimality test cor- 
responds a characteristic computational procedure used in solving linear- 
programming problems. 


§ 2. General simplex procedure 


2-1. We now consider the general procedure for solving the linear- 
programming problem in the nondegenerate case by means of the simplex 
method. 

We remind the reader thata support program of problem (1.1)- 
(1.3) is said to be nondegenerate if the number of its positive com- 
ponents is m A nondegenerate linear-programming problem 
is a linear-programming problem whose support programs are all non- 
degenerate. 

Let a support program X of the problem whose basis is A,, As, ..., As, 
be given. The program is first tested for optimality. This can be done by 
either the first or the second form of the optimality test. In both cases 
we have to compute the A,-values (/=1, 2, ..., 2). In the first case A, are 
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calculated from 


Ay=2)—-0,= Dentiy—ep (2.1) 


where x,, are defined by (1.4). When the second form of the test is used, A, 
are determined from 


Ay=2,—¢,= Baidi—ep (2.2) 


where A=(A,, ..., A) is the solution of system (1.10). 

When testing a support program for optimality, one of the following three 
cases may arise: 

a) A,>0 for all j¢/, (for j€/y z,=¢,, so that in the first case A,>0 for 
all j from 1 to a2). 

b) A,<0 for some single j, and all the x,;,<0 (i=1, 2, ..., m) corresponding 
to this / are nonpositive. 

c) A,<0 for several j/ and for each of these j at least one of the x;, is 
positive. 

In the first case we have, from the optimality test of support programs, 
that X solves the linear-programming problem. 

We shall now show that in case (b) the linear-programming problem is 
unsolvable, whereas in case (c) a method can be indicated for passing from 
the support program X to a new support program closer to the optimum. 

2-2, Let us now see how the linear form (1.1) changes when we pass to 
anew program X(6) formed from program X according to the following 
rules: 

1. the j-th component of program X(6) is taken equal to some positive 
number 6(j is a fixed index of an extrabasis variable of program X): «x, (6)=6; 

2. the remaining extrabasis variables of program X are taken equal to 
zero (x, ($)=«x,=0 for s¢/y, s # J); 

3. the basis components of X(#) are chosen so that the vector X(6)= 
= (x, (6), ..., x, (6)) defines a feasible program of the problem, i.e., the num- 
bers x,(0) (/=1, 2,..., 2) satisfy restraints (1.2)-(1.3). 

In the following we shall refer to this transition from program X to pro- 
gram X(6) as the elementary transformation associated with the 
vector A,. 

Let us now write the relationships specifying the elementary transfor- 
mation. From the definition of the support program X and from the assump- 
tions made onthe structure of program X(6) we have 


2 tnAn=8, (2.3) 
2i*i0 (8) As +4,=B. (2.4) 


We recall that any restraint vector may be expressed in terms of basis 
vectors 


Ay= 2ixi/Ay- (1.4) 
From (1.4) and (2.3) we have 
Di (%e—O%;)) As, +04,= B. (2.5) 


Comparing (2.5) with (2.4) and since the solution of this system is unique 
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(the vectors A,, ..., A, constituting the basis of the support program X are 
linearly independent), we obtain equalities defining the elementary trans- 


formation: 
Xj, (6) = x;, —bx; ,. (2.6) 


Here i specifies the position of the vectors A,, in the basis of the support 
program X (x;,=xs,, x;,(6)=<s, (6)). We recall that according to the definition of 
the elementary transformation associated with vector A, 


x,(§)=x,=0 for t@ly, t#/ x, (8) = 5. 


The elementary transformation is thus well-defined, 

All the x,,— the basis components of program X—are positive. Hence, 
as long as § is small enough for all the ~,, (6) in (2.6) to be nonnegative, the 
vector X(§) satisfies conditions (1.2), (1.3) and as Such is a feasible pro- 
gram of the linear-programming problem, 

Using the components of the feasible program X(@) we easily calculate the 
corresponding value ‘of the linear form (1.1): 


L(x (4)} = 2 Cs;Xj_ (9) + ¢8= »2 Cs, X;5-—9 (2 C5,X;,—C,). 


Applying (1.5) and (1.6), we obtain 
L[X(8)] =L(X)—6A,. (2.7) 


Equality (2.7) shows that the effect of elementary transformation of program 
X on the value of the linear form is determined by the sign of A; If 4,<9, 
the value of the linear form is increased; if A,>0 it is decreased; if A, =0 
the linear form remains unchanged. 

2-3, We now introduce several terms useful in the following. 

The parameter: A,, whose signs determine the direction of change of the 
linear form under the elementary transformation associated with the vec- 
tor A,, shouldbe called evaluations of the restraint vectors A, 
with respect to the given basis (since the coefficients c, of the 
linear form also enter the definition of Aj it would be more exact to call 
the parameters A, evaluations of the augmented restraint 
vectors A, with respect to the given basis). The usefulness 
of this term is obvious from the following considerations. Elementary 
transformations associated with vectors corresponding to A4,<0 and A,>0 
respectively increase and decrease the value of the linear form. For 6=1 
the change in the value of the linear form is exactly 4, The parameters A, 
thus evaluate the change which will take place inthe value of the linear form 
when the vector A, is introduced into the basis, It is this fact that is re- 
flected in the proposed name of the parameters A, According to the pre- 
ceding concept of basis-vector evaluation it would be better to define thus: 
A,=¢c,—z, instead of A,=z,—c, Nevertheless, to avoid all possible confusion, 
we shall keep to the latter definition which, by now, has become accepted in 
the literature on linear programming. So, A,=z,—c, This will also enable 
us to unify the computational procedures, 

When the second form of the optimality test is applied, A, are calculated 


from ae 
Ay= 2a,hi—6y 


The parameters 4, will, henceforth, be called evaluations of prob- 
lem restraints with respect to the given basis. 
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To justify this term, let us recall the definition of the dual propiem: 
the problem concerned with the minimization of the linear form s by, sub- 
ject to the conditions > a,,y;==c, is dual with respect to problem (1. 1)~(1.3). 


It follows from the second form of the optimality test for support programs 
that the parameters 4; associated with the basis of the support solution X* 
satisfy the restraints of the dual problem 


m™ 
A, = Diaijh;— ec, > 0. 


i= 


On the other hand, for an optimal support program X* 


by e a s bs sh e e id e dd e hd e 
L({X*)= Ose )%1 = 2 Cs. *n0 = D (a4 as.) Xp = Da (Y 21s, Xo) = Ai 5;. 
f=) pal poi f=1 i= pri i= 


Hence, the vector A* =(Aj, ..., Am) SOlves the dual problem (see Chapter 3, 
Lemma 1.2). 

In Chapter 3, 5-5 we interpreted the components of the optimal program 
of the dual problem as evaluations of the restraints of the primal problem 
(evaluations of the influence of the right-hand sides of (1.2) on the maximum 
obtainable value of the linear form (1.1)). By suitably choosing 4;, all the 
A; corresponding to the basis vectors of the solution vanish. The para- 
meters A; corresponding to any support program play, with respect to the 
basis of this program, the same role as 4; play, with respect to the basis 
of an optimal support program. We observe from the definition of A, that 
the choice of these parameters ensures that all the 4,— the evaluations of 
the restraint vectors of the basis — vanish. 

The above considerations show that the parameters A; can indeed be 
interpreted as the evaluations of problem restraints with respect to the given 
basis. 

2-4, With the aid of (2.6) and (2.7) it is possible to analyze all the cases 
which arise when testing a support program for optimality. 

In case (a), as we have already seen, the program X solves the linear- 
programming problem according to the optimality test. 

In case (b) there exists an index /=& such that A, <0 and all the corres- 
ponding components x,,<0 (i=1,2,...,m), From (2.6), for j=, we see that in 
this case X(6) is a program of problem (1.1)-(1.3) for any 6>0 (x,(6)=0 for 
any ?>0). We observe from (2.7) that this indicates that the linear form 
L(X) is unbounded above in the set of feasible programs of the problem. In 
case (b) the linear-programming problem is thus unsolvable. 

Case (c), however, is the most frequent. In this case A,<0 for several 
j, and for each of these j at least one of the x,;, is positive. We shall show 
that under these conditions, by means of the elementary transformation 
associated with the vector A,(A,<0), it is possible to obtain from support 
program X another support program xX’ the value of whose linear form is 
higher, 

We carry out the elementary transformation of program X with 6 equal 


to the minimum of x,,/x;, for x, >0, i=1, 2,...,m (by assumption such x,, 
exist). 
Thus let 6, = min x;,/x,,, 
é 
Xin>9 
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By assumption, x,>0 for i=l, 2, ..., m. Therefore 6, is positive. Here we 
use the fact that the support program X is nondegenerate. 

Let X’=X(6,). By definition X(6,) is a feasible program of the problem. 
Moreover, we Shall show that X’ is a support program of the problem. 

Let the minimum of x,,/x;,, defining 6,, be attained for i=r. Obviously, 
x; =X;s,(9)=0. We shall sometimes say that 9, is obtained on the r-th position 
of the basis, or, equivalently, on the restraint vector A,. 

According to (2.4), for given §=6, the constraint vector B is a linear 
combination of the vectors A,,, A, ..., As,_y» Ag As,4,)) -++) As, With nonnegative 
coefficients. It can easily be shown that this system of vectors is linearly 
independent. Indeed, we have the following theorem which will be used re- 
peatedly in the following. 


Theorem 2.1. If P,, P,, ..., P, ts a linearly independent system of vectors 
and the vector q-$ C,P,, where t, #0, the system P,, P,, ..., Pry, Q Pray «-- 


.., P, is also linearly independent. 
Proof. Let us assume the contrary. Then, there exist numbers aq,,q,, ... 
.,a,,a, some of which are nonzero, such that 


m 
p? a;,P;+aQ=0. 
ie 
Expressing Q in terms of P,, P,, ..., PB, and substituting we obtain 


m 

p> (a; +a0;) P; +a,P, =0. 

ifr 
The system P,, ..., P,, ..., P, is, by assumption, linearly independent, and 
¢-#0. Therefore, the above relationship holds only if a=a,=...=a,_,= 
=@,,,=...=a,=0, which contradicts our assumption. This completes the 
proof. 

From this theorem we have that A,, ..., As,_,, Ag» Aspayy e+ +) As, 1S linearly 
independent (here the basis vectors constitute the system P, ..., P,, Q=A,, 
and €.=x,,>0). This means that the feasible program X’= X(0,) is indeed a 
support program, its basis comprising the vectors A,,..., As,» Age Aspayr oe +> As: 

The basis of the new support program is, thus, obtained from the basis 
of the preceding support program when the vector A,, occupying the r-th 
position of the basis (the position on which 6, is obtained) is replaced by a 
vector A, with negative relative evaluation. The vector A, occupies the r-th 
position in the new basis. Accordingly, 


; x;,, (§,) for i#r, 
via 0, for i=r. 


Applying (2.6), we obtain expressions for the basis variables of the new 
support program: 

; Xip—98,Xi—g, ET, 

via 6, i=r, 


(2.8) 


Observe that in the nondegenerate problem @, is obtained for a unique i, 
i.e., only for a single position of the basis. Otherwise the support pro- 
gram xX’ would be degenerate, since it would have had less than m positive 
components. 

From (2.7) 

L(X’)=L(X)—6,A,. 
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By assumption, A,<0. Moreover, we have seen that §, is positive. There- 


fore, 
L(X’) > L(X). 


In case (c) we thus may transfer from the initial support program X toa 
new support program X’ whichiscloser tothe problem's solution, The transi- 
tion to the new program increases the linear form by —6,A,. Successive 
transitions from one support program to another are effected until either a 
solution is obtained, or unsolvability is established. 

Each transformation from a given support program to another constitutes 
an iteration (step) of the simplex method. The number of iterations 
leading to a solution or to the proof of unsolvability is finite. Indeed, to 
each support program of the nondegenerate problem there corresponds a 
characteristic system of m linearly independent restraint vectors consti- 
tuting its basis; there are altogether 2 restraint vectors. Therefore we 
have at most C; different bases. The number of different support programs 
of the problem is thus also finite. For each support program the value of 
the linear form is uniquely determined. In the nondegenerate case each 
successive iteration increases the value of the linear form and, consequent- 
ly, a basis which has once been used in an iteration will not be obtained a 
second time. Therefore, after a finite number of iterations the simplex 
method yields a solution (case (a)) or establishes unboundedness of the linear 
form in the set of feasible programs (case (b)). 

Elementary transformation of support programs is fairly particular. 
Nevertheless, we have seen that elementary transformations invariably 
yield an optimal program of the problem in question. Solving a linear- 
programming problem by the simplex method involves determining suitable 
sequences of these transformations. 

2-4, We review briefly the sequence of operations involved in each se- 
parate iteration of the simplex method. 

Each iteration consists of two stages. In the first stage the given sup- 
port program is tested for optimality. This stage leads to one of three 
possibilities (cases (a), (b), (c)). For case (a) or case (b), the process termi- 
nates. In case (a), the program in question is optimal. In case (b), un- 
solvability of the problem is established. For case (c), we proceed with 
the second stage of the iteration — determination of an elementary trans- 
formation which leads to a new support program with a higher value of the 
linear form. In the second stage a vector A, with a negative evaluation 
(A,<0) is chosen and the position r to be occupied by this vector in the basis 
is determined. The vector A, is eliminated from the basis. Thus, the new 
basis comprises the vectors of the old basis with the vector A, replacing 
A; In the second stage all the parameters necessary for the optimality test 
of the new support program are computed. This new support program is 
then tested for optimality in the first stage of the next iteration. 

All the arguments of the present section were stated for an linear-pro- 
gramming problem requiring maximization of the linear form (1.1) subject 
to conditions (1.2), (1.3). There is no need for a separate discussion in 
the case of minimization. Any problem involving minimization of the linear 
form 


L(X) = Rep, 


subject to certain conditions is reducible to maximization of the 
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linear form 
nm 


L(X)=— Yejx, 


under the same conditions. 


§ 3. Examples 


3-1, We give below two simple examples illustrating the computational procedure for solving linear- 
programming problems by the simplex method using only formulas and results of the present chapter, More 
compact computational procedures are given in the next chapter. Here we consider only the principles of 
establishing an optimal program, not the technique of problem solution. The solution, therefore, is given with 
reference to the first form of the optimality test. 

Example 1, Maximize the linear form 


L (X) =5x,—x,— 2x, +5, + 5x, — x, 


subject to the conditions 


— 2x, +5x, +, =10 

Xi % +X, == | 

X, + 2x, +2, = 6 

10x, —3x, +x, =15 
6 


Xx ;== 0, j=1, 2, ea 


Solution, The restraint vectors and the constraint vector are, respectively, 


—2 5 | 
_ 0 
A, = ’ A,= ' ’ A,;= 0 ’ A,= 0 ) 
10 —3 0 0 
0 fo 10 
ee O Ve poy Ox gut 
t Ses } ® oe 0) r — 6 
0 \s 15 


It can be seen that as an initial support program we inay take 


X=(0, 0, 10, 1, 6, 15). 
For this program L (X)=0. 

According to the general simplex procedure, we express A, and A, in terms of As, A,, As and A, 
which form the basis B of the initial support program. These vectors, respectively, occupy the positions 1, 
2, 3, and 4 in the basis. For the sake of uniformity, we should, in accordance with the notations of § 1, take 
A,=Ayz,, Ay=Ay,, As = Ag, Ag=As,. The expansion coefficients, x;, and xj, of the vectors A, and A, ex- 
pressed in terms of the basis vectors satisfy the equations 


The basis vectors Ag, are unit vectors, The components x;, and x, are, therefore, equal to the corres- 
ponding components of the vectors A, and A,. The coefficients x;,; for j€/y are, obviously, equal to unity 
for j=s,; and to zero for { #s;. The matrix || x;,|| thus coincides with the restraint matrix ]|@;,|| (given 
different positions of the vectors in the basis, the sequence of rows in |j*;,|! and || a;,|| would be different): 


—2 51000 
1—10100 

xy l= 1 20010 =||a;,\}. 
10 —30001 


We now arrange the components of the support program X, which are the basis variables x;,, according 
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to the positions of the vectors Ag, in the basis: 
Xyg= 10, Xy—=1, Xy9=—6, xy = 15. 
The parameters z; and A, are computed from (1.5) and (1.6). We have 


z2=(2,, ---, 2)=(4, —2, —2, 5, 5, —I), 
A= (A,, ait Ay A.) =(—1, —I, 0, 0, 0, 0). 


We see that A, and A, are negative and that to each correspond several positive x;, We thus have case (c), 
i.e€,, we may proceed, by means of the elementary transformation, from support program X to a new support 
program X’ at the same time increasing the value of the linear form. 

Either A, or A, may be introduced into the new basis B, (A,=A,=— 1). Let us assume that A, is intro- 
duced into the basis. To establish which of the basis vectors should be eliminated, we calculate the ratios 
Xjolxj, for xj, > 0. Wehave x,, <0, whereas for i=2, 3, 4 the ratio is equal to 1; 6; */,, respectively. 

The least ratio (= 1) corresponds to i=2. Hence, 6,=I and the vector A, is introduced into position 2 of the 
basis (the position on which 6, is obtained) in place of the vector A,,=A,. The basis components Xig=*lo (6,) 
of the new program, re-enumerated according to the positions of the basis, are computed from (see (2.8)) 


Bi -{ Xjig—1x;, for it #2, 
2. 6,=1 for i=2, 


The component n=%, is again zero, The elementary transformation thus leads to a new support program: 
X’ =(1, 0, 12, 0, 5, 5). 


On this program the linear form has increased from L(X)=0 to L(X’)=1. 
We now express A, and A, (vectors not appearing in the basis B,) in terms of the vectors B;. As before, 
we have to solve two systems of equations: 


W 
A,= DELAY fe 
f= 
: ? 
A,= p> x1,As;. 
=1 


In the new basis the vector A, is in position 2, The rest of the vectors of B, remain in the same positions as 
in the preceding basis: 
A;,=As, As,= A, A,,=A,, As, = Ag. 


Solving the two systems and since x, j=O7 for 7 Ely, we obtain (8) is the Kronecker delta equal to 1 
when j= and to 0 when j ¥ ¢) 
0 31 200 
, 1—10 100 
ltull=Io 30 110 
0 70-1001 


As a check we may also determine the basis components of X’ from 


m ’ 
ely, p> 


To the new support program X’ correspond the vectors Z’ and A’, Their components are computed in the 
Same way as the components of the vectors Z and A: 


z' =(5, —3, —2, 6, 5 —1), 
A’ =(0, —2, 0, 1, 0, 0). 


Here A, is negative, Moreover, to A. correspond several positive Ke: We thus may, by means of the ele- 
mentary transformation, increase the value of the linear form, Subsequent iterations are carried out according 
to the same rules as the preceding ones. Omitting the intermediate computations, we arrange the components 
of the vectors X‘), Z) and A's) and the corresponding values of the linear form L in the foilowing tableau 
(Table 4,1). We observe that all the components of the vector A) are nonnegative, According to the opti- 
mality test, we have obtained an optimal program X*— a solution of the linear-programming problem. 
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TABLE 4.1 


101 /23 


—13/7 20/23 


L (X) 17/7 | 93/23 


3-2. We now give an example where the process terminates in case (b). 
Example 2. Maximize the linear form 


L (X)=3x,+ 4%, 


subject to the conditions 2x, —3x, +x 
1 2 3 


=I, 
—5x, + 2x, +x,=1, 
x;=0, i=}, ..., 4. 


Solution. It is easily seen that the vector 


X=(0, 0, 1, 1) 


may be taken as the initial support program. On program X, L(X)=7. The restraint vectors and the con- 
straint vector of the problem are, respectively, 


sa(B)s Ae (Bs Am(8)s A Q)s 2), 


The initial basis, B, comprises the unit vectors A,=A,, and A,=A,,. Therefore 


2-310 
—5 201 


tI xa =H aili=| 


Enumerating the basis components of the support program X according to the positions which the corres- 
ponding restraint vectors occupy in the basis we obtain 


Xy=Xy=—1, xyHrAyQ_=!. 
Applying (1.5) and (1.6), we compute the components of the vectors Z and A: 


Z=(—14, —1, 3, 4), 
A=(—14, —1, 0, 0). 


To the negative evaluation A,=»—14 corresponds a single positive x;,, namely x,,=2. Therefore, A, 
is introduced in the basis in the first position of the basis, since the minimum ratio 


Xio __ *lo 
Xin, Xin 


is obtained for 4=1. The vector A, occupying the first position of the basis is eliminated. The component 
x, of the new support program is 
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The other components of X’ are computed from (2,8), Here x,=.x, is again zero, since Ay, as before, does 
not appear in the basis, 
We have thus obtained a new support program 


ee | 7 
x -{5° 0, 0, 3h 


The value of the linear form corresponding to the support program X’ is L (X’)= 

The basis B, of the support program X’ comprises the vectors A, and A,. According to the positions of 
these vectors in the basis B,, we have A,=Ag,, Ay=Agy, and x. =Xip x =Xiq. We now express A, and Ag, 
which do not appear in B,, in terms of the vectors of the new basis, The components Xi: and Xis are deter- 


mined from the equations 


a 
A,= Di *aAsp 
#=1 
. a 
A,= Ee 
f=1 
Solving the two systems and since x,, = oi for 7] € ly,, we obtain 
1 —*l; ‘hs 0 
laill=[ —"h th! 


Zz’ =(0, —22, 10, 4), 
A’ =(0, —22, 7, 0). 


Hence 


We see that AY <0 and all the components Xi <0. This relationship between the parameters A, and 
xy corresponds to case (b), when the linear form has no upper bound in the set of feasible programs of the 
problem. The problem in question is thus unsolvable, Indeed, the point 


X=(t, t, 144, 1432) 


satisfies the problem restraints for any ¢2==0. Here 


L(X)=7+4+ 15¢ 


increases to infinity as ¢ increases. 


§ 4. Geometrical interpretation of the simplex method 


4-1, All the elements of the simplex method have a simple geometrical 
interpretation. In Chapter 2, §5 we advanced two geometrical interpreta- 
tions of the linear-programming problem. The reader is advised to reread 
the section dealing with the geometrical images associated with the alge- 
braic description of the present chapter. The geometrical interpretation of 
the method will be given in terms of many-dimensional spaces. To under- 
stand the geometrical illustration well, the reader may confine the analysis 
to the two- or three-dimensional case. The examples in 4-2 and 4-4 are 
especillay designed for this purpose. 

Different geometrical interpretations of the problem lead to different 
geometrical interpretations of the method. 

According to the first geometrical interpretation of the linear-program - 
ming problem, restraints (1.2)-(1.3) define a convex polyhedron (or an un- 
bounded convex polyhedral set) in the space of variables x,, ...,x,, the di- 
mensionality of this set not exceeding 2a—m. This polyhedral set is con- 
tained in the intersection of the hyperplanes defined by restraints (1.2) of 
the problem. 

To a support program X corresponds a vertex of the polyhedral set. 
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This vertex, like the support program, will be denoted by X. The vertex 

X is formed by the intersection of hyperplanes satisfying restraints (1.2) and 
hyperplanes corresponding to the nonzero components of the support pro- 
gram. In the nondegenerate case all the basis variables are nonzero. A 
Support program contains ra—m extrabasis components. Correspondingly, 

in the nondegenerate case, the number of independent hyperplanes inter- 


secting at the vertex X is 
m+(n—m)=an. 


Take one of the extrabasis variables x, and replace the condition x,= 
by the constraint x,>0. If all the other extrabasis variables retain their 
zero values, the locus obtained describes a ray "inside'' the polyhedral set 
(where x,>0). We shall say that this ray corresponds to the restraint vec- 
tor A,. 

The locus of all points for which the linear form 


me 
ft 
) 


maintains a constant value is a hyperplane. This will be called the linear- 
form hyperplane. The vector 


C=(c,, Cy ones c,) 


specifies the direction in which the hyperplane should be displaced to in- 
crease the value of LZ. The linear-form hyperplane generates two half- 
spaces in the x,, x,,..., x,-space. The half-space containing the vector C 
will be called the upper half-space; the other half-space is known as 
the lower half-space. 

Geometrically, elementary transformation of the support program X as- 
sociated with the vector A, is a translation from the vertex X along the ray 
corresponding to the vector A, Any edge of the polyhedral set issuing from 
the vertex X is the intersection of some a—1 independent boundary hyper- 
planes containing the point X. Therefore, in the nondegenerate case, to 
any such edge there corresponds a ray determined by one of the vectors 
A, j/¢/y. On the other hand, in our case the intersection of the ray corres- 
ponding to any vector A, /€/y, with the polyhedral set contains a segment 
(6, >0) and, consequently, constitutes an edge of the given set. 

Thus, when the program X is nondegenerate, there is a one-to-one cor- 
respondence between the vectors A, /¢/,, and the edges of the polyhedral 
set issuing from the point X. We recall] that edges of the polyhedral set may 
be either bounded or unbounded, The former are segments joining two 
vertices of the set, the latter are rays issuing from certain vertices. 

We may now say that each set of elementary transformations with 0 <6 <4, 
is geometrically equivalent to translation along an edge of the polyhedral 
set. If §,< oo the corresponding edge is bounded. If 6,00 the edge is not 
bounded. 

For fixed 6, we move to the point X(§), which modifies the linear form 
by Af. 

We now give in geometrical terms all the elements of a single iteration 
of the simplex method. 

Inthe first stage of an iteration the support program X is tested for 
optimality. We draw through the vertex X the linear-form hyperplane and 
select all the edges of the polyhedral set issuing from the point X and 
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contained in theupperhalf-space. If there are no such edges, i.e., all the 
edges containing the vertex X are below the linear-form hyperplane, the 
support program X is optimal (case (a)). Geometrically it is obvious that if 
all the edges issuing from the vertex X of a polyhedral set are below the 
linear-form hyperplane, the entire set is below the hyperplane. This 
proves optimality. 

Assume now that some edges of the polyhedral set lie in the upper half- 
space. Two possibilities arise. If among the edges in the upper half-space 
there is an unbounded edge (ray), the linear form obviously has no upper 
bound (case (b)). If, however, all the edges of the polyhedral set contained 
in the upper half-space are bounded (all the edges are segments), we may 
proceed to the second stage of the iteration, i.e. , effect a translation along 
one of these edges thus increasing the value of the linear form (case (c)). 
Parallel translation of the linear-form hyperplane is continued until it inter- 
sects the other end point of the edge, the adjoining vertex x’. At xX’ one of 
the basis variables of the old basis vanishes. The vertex xX’— the image of 
the new support program X’— is thus the intersection of the edge-forming 
hyperplanes with the hyperplane corresponding to the old basis variables 
which has now vanished, Obviously, 


L(X’) > L(X). 


Proceeding with this parallel translation of the hyperplane L(X)=const from 
a given vertex of the restraint polyhedron to another (adjoining) vertex, the 
value of the linear form is increased with each iteration. Since the number 
of vertices of the restraint polyhedron (polyhedral set) is finite, the maxi- 
mum of the linear form is established after a finite number of steps. It is 
also obvious from geometrical considerations that unsolvability of the prob- 
lem (unboundedness of the linear form in the domain of its definition), if 
the problem be indeed unsolvable, is also found after a finite number of 
steps. 


4-2. We now illustrate the geometrical interpretation applying our reasoning to the examples given in § 3. 
We have already noted that the restraint polyhedron is contained in the intersection D of hyperplanes 
corresponding to the equality restraints of the problem, Introducing a system of coordinates in D we obtain 
the equivalent linear- programming problem with all restraints represented by inequalities, The equivalent 
problem can be obtained, for example, by expressing the basis variables of some support program in terms 
of the extrabasis variables and eliminating the basis variables from the restraints and from the linear form of 
the problem, Applying this procedure to the restraints of Example 1, we obtain 


%,=10+2 x,— 5x,, 
xX,= 1— x,+ 
X¥,= 6— x,— 2x,, 
xX, = 15— 10x, + 3x,. 
We now state the equivalent problem. 
Maximize the linear form 


L (X)=x,+%, 
subject to the conditions 
10+2 x,—5x,>0, 
I— x,+ x,>0, 
6— x,—2r,>0, 
15— 10x, + 3x, >0, 
x,=>0, x,>0. 


The domain of definition corresponding to the linear form of the equivalent problem (and consequently 
of Example 1) is shown in Figure 4.1, The sides of the restraint polygon are segments of the straight lines 


x,=0; x,=0; 
¥,;=10+4+2 x,—5x,=0; x,=6 — x,—2x,=0; 
xX=1— 2x,+ x,=0; x,= 15— 10x, + 3x, =0. 
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Striation along the line indicates the half-plane in which the corresponding variable is positive. The lines 

L (X) =const (x,+-x,=const) are parallel to the line MN in Figure 4.1. The arrow OC indicates the direc- 
tion in which the line MN should be translated to increase the value of the linear form, The initial support 

program corresponds to vertex O of the restraint polygon. The point O is the intersection of lines x,=0 and 
x,=0 corresponding to the extrabasis variables. The value of the linear form at this vertex is zero. 


z,tz-0) ® 


a 


+ 


be = = oe = oe 


i 
iW’ 
FIGURE 4,1 FIGURE 4.2 


We observe from Figure 4.1 that all the sides (edges) of the polygon issuing from the vertex O are con- 
tained in the upper halfplane. Hence, translation to any of the adjoining vertices increases L. Elementary 
transformation of the support program X to program X’ corresponds to translation from vertex O to vertex P 
of the restraint polygon. Translation is effected along the ray defined by the equation x,=0, x,=.0. The 
ray OP thus corresponds to the restraint vector A,. At vertex P, the basis variable x, vanishes. Hence, the 
point P is the intersection of the lines x,=0, x,=0. The vertex P is the image of the support program X’. 

Transition from program X’ to program X" and from program X”’ to program X’”’ is realized by translation 
from vertex P(x,=0, x,=0) to vertex Q(x,=0, x,=0), and from this Q to vertex R(x,=0, x,=0). Trans- 
lation is effected along the rays corresponding to the restraint vectors A,(x,==0, x,=0) and A, (x0, *,=0), 
respectively. Transition to each successive vertex increases the value of the linear form. 

All the sides (edges) of the polygon issuing from the vertex R are below the line M’’’N’’’, According 
to the geometrical interpretation of the optimality test, this means that the maximum of the linear form is 
obtained at vertex R. We observe from Figure 4.1 that any parallel translation of the line M’’’N’’’ ensuring 
nonempty intersection of this line with the restraint polygon decreases the linear form L(X). The coordinates 
of the point R thus specify the optimal program of the problem. 

In Figure 4.2 we show the domain of definition of the linear form of Example 2, We see that this domain 
is an unbounded convex polygonal figure. The lines £(X)=const are parallel to the line MN. The vector 
C gives the direction in which the linear form increases, The initial support program X corresponds to vertex 
O of the polygonal figure. The value of the linear form L(X) at the vertex O is 7, Translation to any ad- 
joining vertex increases the value of L. The program X’ corresponds to the vertex P. L(X’)=14. The 
equation of the line M’‘N’ is 

L(X’)=14, 
“ 14x, -+%,=7. 
The ray x,220, x,=0 issuing from the vertex P is an unbounded edge of the polygonal set representing the 
domain of definition of the linear form. This ray is above the line M’N’, According to the geometrical 
interpretation of the simplex method, this establishes unsolvability of the problem, We see from Figure 4,2 
that when M’N’ is translated parallel to the direction indicated by the vector C, it always intersects the 
domain of definition of the linear form. In the process L(X) increases to infinity. We thus see that the 
linear form has no upper bound in the set of feasible programs, 
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4-3, We now give the geometrical interpretation of the simplex method 
corresponding to the second geometrical interpretation of the linear-pro- 
gramming problem. 

We remind the reader that the augmented restraint vectors A, generate 
a convex polyhedral cone K inthe (m+1)-dimensional space of points U=(u,, 
U,,..., Byy,). To restraint vectors A, and to the constraint vector B in the 
U-space correspond points with u,,,=0. A line Q passes through the point 
defined by the constraint vector 8, parallel to the Ou,,,-axis. If the line Q 
does not intersect the cone K, the set of feasible programs is empty, l.e., 
the restraints are inconsistent. 

We shall deal with cases where the set of feasible programs is nonempty, 
i.e. , when the intersection of the line Q and the cone K is not empty. The 
part of the line Q contained in the cone K is the image of the domain of de- 
finition of the linear form in the (m+1)-dimensional space of points U. The 
highest point in the intersection of K and Q (the point of the segment with 
the greatest value of the (m+ 1)-th coordinate) specifies the maximum of the 
linear form L, and the lowest point gives the minimum of L (existence of 
these points is implied). 

In some cases the intersection of the line Q and the cone K is a ray 
directed upwards or downwards. In this case the linear form is unbounded 
above or below on the set of feasible programs. If the entire line Q is con- 
tained in the cone K, the linear form is unbounded in either direction in the 
domain of its definition. 

We shall now consider the geometrical constructions associated with 
each iteration of the simplex method, 

Let the restraint vectors A,, A,,..., A, constitute the basis of some sup- 
port program X. The corresponding augmented vectors 4A,,..., A, are also 
linearly independent. Let II denote the hyperplane generated by the vectors 
A,, .»., 4, and passing through the origin. This hyperplane is uniquely de- 
termined by the support program X and is indeed its image in the U-space. 
The hyperplane II divides the augmented vectors A,,,,-.., A, into two 
groups. One group is on the side of the hyperplane II which contains the 
positive semiaxisOu,,,, We shall say that these vectors are above the hyper- 
plane II. The second group contains all the vectors A; which are below II. 

As before, we shall use the same symbol to denote vectors and the cor- 
responding points, Let the lines passing through the points A, and A, (paral- 
lel to Ou,,,) intersect the hyperplane II at the points Aj, and the line Q, at 
the point B*, 

We now express the lengths of the segments 4j A,, A, A; and B°B in terms 
of the parameters of the simplex method. This will enable us to relate the 
geometrical constructions with the algebraic description of the method 
given above. 

By definition, the point A} belongs to the hyperplane Il. We express the 
vector Aj in terms of the m linearly independent augmented restraint vec- 
tors A; spanning the hyperplane II: | 


m 
Aj = > XA; (4.1) 


The first m components of the vectors A; and A, specify the restraint vec- 
tors A, and A, respectively. We have that (see (1.4)) 


m™ 
as 2 %yAp 
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Therefore 
Xjp=X ype (4.2) 


The length of the directional segment 4jA, is actually the (m+1)-th com- 
ponent of the vector 4;*. 

The last component of the augmented restraint vector A; isc, We 
therefore have from (4.1), (4.2), and (1.5) 


m 
and 7 = 
|A,A5| =|[4,4,|—|AjA| = ¢,—z;=—A,. (4.4) 


The segment B°B determines the value of the linear form for the given 
support program: 
\B°B| = L (X). (4.5) 


Thus, for the augmented vectors of the first group, those above the 
hyperplane II, A;<0. For the vectors A, of the second group, A,>0. 

Let A, be an augmented restraint vector not contained in the hyperplane 
Ii. Consider the (m+ 1)-dimensional cone spanned by the vectors A,, A,, . 
.-+.A4,, 4, We shall denote this cone by K, (j being the index of the vector 
A,¢Il). The intersection of the line Q and the cone kK, comprises points 
which are geometrical images of programs which are obtained from X by 
the elementary transformation associated with the vector A,(0<6<6,). In 
the nondegenerate case the point X is situated inside the face spanned by the 
vectors A,,..., A,- The intersection of Q and K, for any A,€II therefore 
contains a certain segment. In the degenerate case the intersection of Q 
and K; may contain only one point (Q tangent to K;,). 

In the first stage of the iteration the support program is tested for opti- 
mality. The geometrical test reduces to deciding whether there exist aug- 
mented vectors above the hyperplane If. If the first group of augmented 
vectors is an empty set, the support program X is optimal (case (a)). In 
this case the entire cone K is below the hyperplane II. The maximum value 
of the linear form is determined by the length of the segment B°8, 

Let the first group of vectors A, be nonempty, i.e., there exist aug- 
mented restraint vectors above the hyperplane I]. Here two possibiliites 
may arise corresponding to cases (b) and (c), respectively. 

Geometrically, case (b) obtains when there exists a vector A, above the 
hyperplane II such that the intersection of K, and Q is a ray issuing from 
the point X. We observe that this is possible if and only if the cone K, con- 
tains the halfaxis Ou,,, (see Exercise 3). If the first group of augmented 
restraint vectors is nonempty, and none of the kK, contains Ou,,,, we have 
case (c). In this case we may pass from II to Il’, corresponding to a suc- 
cessive support program X’. 

The hyperplane II’ intersects the line Q at the point B’ above the point 8°. 

The transformation from II to II’ is effected as follows. Let the vector 
A, be above the hyperplane II. We form the cone k,. As we have already 
observed, in the nondegenerate case the intersection of K, and Q contains 
a certain segment. In case (c) the intersection of K, and Q may not be a 
ray. It therefore comprises a segment whose end points are the highest 
and the lowest points of intersection of K, and Q. The point B° is the 


* Here the “length” ||AB|] of a directional segment AB is taken as the length of the segment AB with the 
corresponding sign. |AB|>0 (|AB]<0) if A is above (below) B in the sense of the Ougy,-axis. 
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lowest point of the segment. Let B’ denote the highest point of the segment. 
Observe that the (m+ 1)-dimensional cone kK, has (m-+1) m-dimensional faces. 
Each of these faces is a cone spanned by some m vectors of the system 
A,,+.+, Ap, A, Let the face of the cone K, not containing the vector A, (/=1, 
2,..., m, k) be denoted by T,. In the nondegenerate case, the point B’ lies 
inside some face of the cone k,, say I’,,. 

Let I’ be the hyperplane containing the face [,,. The hyperplane II’ is the 
geometrical image of the new support program xX’. Obviously, the basis of 
program X’ comprises the vectors A,, A,,..., Ap-4. Apayss +++ Ag: Ay Which 
enter the system spanning the face [,,. The line Q intersects the hyper- 
plane JI’ at point B’. The point B’ is above the point B’, so that the program 
X’ corresponds to a higher value of the linear form of the problem than 
problem X. This completes the iteration, The successive iteration is ef- 
fected according to the same rules. We again check whether the hyperplane 
Il’ is above the cone K. If the answer is in the positive, the hyperplane II’ 
specifies an optimal program, and the point B’ determines the maximum 
value of the linear form, Otherwise, we should pass from hyperplane II’ to 
hyperplane II’ following the same rules as in the transformation from II to 
Il’. In the nondegenerate case we invariably obtain a hyperplane II* above 
the cone K, or establish unsolvability of the problem. 


4-3, Let us now illustrate the solution of the following problem by geometrical constructions. 
Maximize the linear form 
L=10x, + 8x, +7x, + 16x, + 21x, 
subject to the conditions 
4x,+- 2x,+ 5x, +10x,+ 5x,= 6, 
9x, + 10x, + 12.5x, + 18x, + 16.5x, = 14, 
x; => 0, j=1, 2,..., 5. 


Figure 4,3 shows restraint vectors A,, the constraint vector B, the corresponding augmented vectors, and 
the line Q passing through the point B parallel to the Ou,-axis. We see that the line Q intersects the convex 
polyhedral cone spanned by the augmented restraint vectors. Hence, the set of feasible programs is nonempty. 


u; 


FIGURE 4.3 
The vectors A, and A, are linearly independent, and the vector B lies inside the cone (here, plane angle) 


spanned by these vectors, This indicates that the constraint vector can be written as a linear combination of 
the vectors A, and A, with positive coefficients, Thus, it is certainly justifiable to take the restraint vectors 
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A, and A, asthe basis of the initial support program X. The two-dimensional cone spanned by the vectors A, 
and A, is striated in Figure 4.4, The hyperplane II (in our example, ordinary plane) is defined by the two- 
dimensional cone spanned by the augmented restraint vectors A, and A,. In Figure 4.4 this cone is alsostriated, 


FIGURE 4.4 


By construction, the points A, are ata distance ¢, from the u,0u,-plane, In our problem 
C={c;}=(10, 8, 7, 16, 21). 


Perpendiculars dropped from the points A; to the u,Ou,-plane intersect II at points Aj, respectively, As we 
have seen, the "lengths" of the segments Ay A j are, respectively, equal to zy, and 


|B°B| = L (X). 
In our case 
206 48 41 
rate) (28. $1184), 
L(X)= 


The “length” of the segment 4, Aj is ¢j-—z,;=: —A,;, In our example 


ie ay ce), 
where A,, A, and A, are negative. 

Consequently, thepoints A;, A, and A, are above the plane II, The point Ay, is the highest of the set 
(A,<A, j=l, 2, 3, 4). Itis, therefore, expedient to introduce the restraint vector A, into the basis, 

We observe from Figure 4,4 that the constraint vector B lies inside the cone (plane angle) spanned by the 
vectors A, and A,, but outside the cone spanned by the vectors A, and As. This indicates that the coefficients 
in the expression of the constraint vector B in terms of A, and A, are positive, whereas those in the expres- 
sion of B in terms of A, and A, may also be negative. We thus conclude that A, should be eliminated from the 
basis. 

We arrive at the same conclusion also from the following considerations, which have been previously given 
for the general case. The line Q intersects the three-dimensional cone K, spanned by the vectors A, and A 
(corresponding to the basis of the support program X) and the vector A, (corresponding to the restraint vector 
introduced into the basis) at two points: B® of the two-dimensional face I',, defined by the vectors A, and A 
and B’ of the face I'y, and defined by the vectors A, and As. The point B® belongs to the plane II generated by 
the vectors of the initial basis, The point B’ is inside the plane Il’ spanned by the vectors of the new basis, Again 
A, should be eliminated, 
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Similar constructions give the points A; and B’. Here 
—A’ =C—Z' = {e,—z;} ={|Ay AA—|A's A = 
ae 12 472 106 
=4[4 - i = (i — ap — ig: 00 ): 


1136 
L(X')= ~7e- 


We observe from Figure 4,5 that at this step only one point Aj, namely A,, is above the plane II’ (A,<0). 
Hence, to increase the value of the linear form the vector A, must be introduced into the basis, Reasoning as 


FIGURE 4.5 


FIGURE 4.6 


above we conclude that the vector A, should be eliminated from the basis (the constraint vector B lies outside 
the angle spanned by the vectors A, and A,). The vectors A, and A, define the plane I” (Figure 4,6) above 
the cone K. The support program X” corresponding to the basis (A,, A,) is thus a solution of the problem. 
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Expressing the constraint vector B in terms of the basis vectors, we obtain the optimal program 


_ (29 
x= (5, 0, 0, 0, 55) 


The line Q intersects the plane II’ at the point 
wy =HL (X=. 


This is the maximum value of the linear form. 


§ 5. The case of bilateral restraints 


5-1, Consider the following linear-programming problem. 
Maximize the linear form 


(= Lepr, (5.1) 
subject to the conditions 
aA j2;=B (5.2) 
a,x, <P; j=1,2,...,2. (5.3) 
The matrix |l@;jlnn=(A,, 4,,+--, 4,) is assumed to be of rank m, 


Problem (5.1)-(5.3) differs from the general linear-programming prob- 
lem (1.1)-(1.3) written in canonical form only in restraints (5.3) which, 
generally speaking, impose upper and lower bounds on the variables, 
Problems with bilateral restraints are often encountered in practical appli- 
cations and they therefore deserve special discussion. 

We shall not assume that all the variables of the problem are bilaterally 
constrained by (5.3); some of them may have only one bound. [If the 
variable x, is bounded only above, a,=—oo; if x; is bounded only below, 
B,=oo. In particular, if a,=0 and B,=oo, problem (5.1)-(5.3) reduces to the 
general] linear- -programming problem in canonical form. 

Problem (5.1)—(5.3) can be reduced to canonical form. To this end it 
suffices to replace the variables x, by x;=x,—a,, j=1,2,...,2, imposing non- 
negativity requirements on the variables x; for j=n+4-1,...,2”, (Here we as- 
sume alla, and 6; to be finite.) 

Thus, a problem with bilateral restraints can be solved in two stages. 
Problem (5.1)-(5.3) is first reduced to canonical form and then solved by 
the method described in the preceding sections for problem (1.1)-(1.3). This 
method, however, is not practicable. The canonical form of the problem 
with bilateral restraints has 2n variables and 2+ m equality restraints. In 
solving this problem we therefore have to deal with (n+ m)-dimensional re- 
straint vectors. Each iteration becomes most cumbersome. 

If each variable of problem (5.1)-(5.3) is unilaterally bounded (above or 
below), the problem is easily reduced to canonical form by the following 
substitution of variables 
; ee if x,>a,, 

By—*,, if x, <B,.- 


The additional restraints distinguishing the problem in question from 
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problem (1.1)-(1.3) are, thus, very simple in structure: 
x, B,, 


where x, is any bilaterally restrained variable of problem (5.1)-(5.3). The 
simple structure of the additional restraints indicates that the simplex 
method can probably be extended to problems with bilateral restraints with- 
out substantial difficulties. In the following we shall show that this is indeed 
SO. 

Notice that by taking into consideration the specific features of the prob- 
lem and adapting the linear-programming methods to the natural form of 
the problem, the computations associated with the determination of optimal 
programs are as a rule greatly simplified. In the following we shall often 
modify the computational methods of linear programming when applying 
them to various particular classes of problems. This modification is ge- 
nerally best achieved by applying the method taking into account the general 
procedure and the specific features of the problem. We shall do precisely 
this. 


5-2, We first introduce some definitions. The vectors A,=(a,,,..., @,,)' 
j=1,...,a are called restraint vectors of problem (5.1)-(5.3), and the 
vector B=(b,,..., 6,)) is the constraint vector of this problem. 


A feasible program X=(x,, x,,..., x,) of problem (5.1)-(5.3) is called a 
Support program if the system of restraint vectors A, corresponding 
to the components x, for which 


is linearly independent. 

Observe that this definition of a support program is fully consistent with 
the concept of a support program introduced in Chapter 2, 4-2 for an arbi- 
trary linear-programming problem. The proof of this is left to the reader 
(see Exercise 4). 

Let X be a support program of problem (5.1)-(5.3). Consider the set 
of all the restraint vectors to which the components x, of program X satis- 
fying inequality (5.4) correspond. The system of m linearly independent 
vectors containing this set is called the basis of the support program X. 
The components of the support program corresponding to the basis vectors 
will be called basis variables, the remaining components being 
extrabasis variables, Anextrabasis variable x, is, obviously, equal 
to one of the limiting values of the corresponding program component 
(a, or B,). 

9-3, We now formulate the optimality test which is the basis of solving 
problems with bilateral restraints. 

Let X=(x,, ..., *,) be a support program of problem (5.1)-(5.3), its basis 
comprising the vectors A,, A,,,...,4,,, As previously, let /, denote the set 
of indices of the basis vectors. Applying the notations of §1, we have 


Ay= 2 xy Ay; J=\1, ) ee n; (5.5) 
= 
z2,>= 2 Xi Cy, j=), / rer A, (5.6) 
=} 
Let, as in §2, 
Mig =X, i=1, 2, , Mm, 
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and let us introduce a vector A, satisfying the equality 
A, = B— yay =2) Xn As, . 
Combining the above relationships with (5.5), we obtain 


Ay= tn j=0, 1, 2,...,2”. (5.5") 


We remind the reader that the first subscript in the coefficients x, (the 
index i) indicates the position of the vector A, in the basis. 
Now let the parameters A, be given by 


— if : 
A =| 2; Cry : a, <= xX,<B, (5.7) 


Obviously, for j€/y, in particular, with a,<x,<f,, 
4, =0. 


In the following we shall see that A, are analogs of the relative evaluations 
of the restraint vectors of problem (1.1)-(1.3). 

Definition (5.7) of the parameters A, makes it possible to state the opti- 
mality test for support programs of problem (5.1)-(5.3) in the same form 
as for problem (1.1)-—(1.3). 

Optimality test. A support program X is a Solution of problem 
(5.1)-(5.3) of 4,20 for all jély. 

Proof, Let X’=(x,,...,%,) be a support program of problem (5.1)—(5.3). 


We have , an, ’ ; : 
L(X = Bo*1 = By +2 CX) p2 ¢ Xf, 
where 2%) are summations over all jé/y for which «,=a,(x,=6,). 
G@ 
Applying (5.3) and the conditions of the optimality test, we obtain 
D2 Cy (x;—a,) <= D7 (*;—a,), 
p2 Cc (x; —B,) 4 p2 z; (x;—B,). 


Therefore 
¢, (%j—%,) <= Siz, (xj; —2,). 
ay! | J 4) Ri | ! j) 
This inequality gives the following estimate for the value of the linear form 
at point X’: ) ; 
L(X’)<— >) cx 2,(xj;—% C,%p. 
(X’) ay f i+ 4 (%) N+ 24 y 
Further, applying (5.6) we have 
m ’ , 
L(X’ . (x; — 2X ; 
(x)= p> Cs, [Xs, + ep tu | i—*,)] +2 oF (5.8) 
We shall now show that 
tut 3 xy (41%) = Hq =Xjo5 b= 1. 2.285 » m. (5.9) 
1x 


Indeed, X’ is a feasible program. Therefore 


>> Au Xn =B— > Ax) (5.10) 
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Multiplying both sides of (5.5) by x;—x, and adding the results obtained for 
jtly to (5.10), we obtain 


3 A, (x, ;—x)) =B—DA)x,=A.. 
2 1» (Xs, + eu (x;—~x,)] >) %j ° 


Let us compare this equality with (5.5') for j=0. The vectors A,, (i=1, 2,... 
.,m) constitute the basis of the support program xX. Linear independence 
of this system of vectors establishes the validity of (5.9). 
From (5.8) and (5.9) we have 


m R 
L(x’ = %,=L (X). 5.11 
( )S Ritakint Bee Pa yh (xX) ( ) 
Formula (5,11) applies for any xX’. This indicates that X is an optimal pro- 


gram of problem (5.1)-(5.3). This proves the optimality test. 
In §1 we established the following relationship: 


2, = Miaip J=i1, y eee _ ft, (5.12) 
where the vector A=(A,,...,4,) is determined from the system of equations 
DSaaj=c; for s€ly. (5,13) 


7=1 


Thus, the parameters z, can be computed either from (5.6) or from 
(5.12). Depending on the formula chosen for computing z,, required in de- 
termining Aj we Shall distinguish, as in §1, between two forms of the op- 
timality test. In the first form of the test, 2, are computed from (5.6), and 
in the second form from (5.12). The second form of the optimality test can 
be stated as follows. 

Optimality test. A support program X=(x,,...,%,) is optimal if the 
vector A determined from (5.13) satisfies the inequalities 


m 
(A, A)= 2aihi> c, tf x,=a,, 
(A, A)= 2 aAi<c, Uf x,=8;. 

9-4 To describe the general procedure for solving problem (5.1)-(5.3) 
it is expedient to extend the concept of elementary transformation intro- 
duced in § 2 to the case of bilaterally restrained variables. Ina problem 
with bilateral restraints the elementary transformation associated with the 
vector A, (jé/y) is defined differently depending on the actual limit (a, or B,) 
with which the extrabasis variable x, coincides. 

Let us consider the two cases separately: 

(1) x=a,, fly, 

(2) x,=B,, J€ly. 

1, In the first case, the vector A, is associated with several transforma- 
tions of the support program X to a program X (6)=(x,(6),...,x,(6)) defined as 


x, +0, if poy, (5.14) 
Xu if ugly, BS. 


We write the conditions under which the vector X(6) is a program of problem 
(5.1)—(5.3). 


Xs, —9x;,, if pP=Si, bos i, 2,...,m, 
x, (6) = 
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Applying (5.5), we have 


241 0.4,= 3 24,4014) — Bay Al= 3x48 


for any 6. Hence, the vector X(6) satisfies restraints (5.2) for any 0. 
We now determine the set of all § for which all the components of the 
vector X(§) satisfy (5.3). 
Since 


the inequalities 


are satisfied if and only if 
0<mb6<p,—a,—B ,—x,. (5.15) 
The other restraints imposed on @ are determined by 
Gs, <= %s,(0) <= Bs, for i=1, 2,...,m. (5.16) 


Obviously, these inequalities need not hold only if x,,#0. 
If x>9, 
Xs, (8) = Xs, — 0%); << xy, (6 > 0). 


Hence, for x,;,>0, the i-th inequality of (5.16) is satisfied if and only if 


—@ 
p88. (5.17) 


If x,,<0, then for 6>0 
Xs, (0) = x5, — Ox, > Xs, 


Hence, for x,,<0, the i-th inequality of (5.16) is satisfied if 
v< Fu *u wn 4B uy (5.18) 
—*iy Xj 


Combining (5.15), (5.17), and (5.18), we see that X(0) satisfies all the 
restraints (5.13) if and only if 


0<6 <),, (5.19) 


where 6 is the least of the three numbers 


Qa) x, —€ 
6 = min——4, 


xy>o | FJ 
60 = min —# Bu 
Zy<o xij 


Until now we have considered m positions of the basis, each position re- 
presenting one of the basis vectors. To simplify notations, we find it con: 
venient to introduce the (m-+-1)-th position in which the vector A,=A, e 
which specifies the given elementary transformation, stands, This addi- 
tional position will be of use in the following also. 
Let 
Sear sh Xmti.0— %j= 5, Ye 


If we take 


a,, for ¥,;>9, i=], ) re m+, 
i= (5.20) 


B, for #,,<0, f=1, 2,...,m+1, 
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the formula for 6, can be written in the following compact form: 


6.= min 2—V, (5.21) 
m4 sey ¥ij 
igigm+i 
Here, as before, x,,=x;, is the basis variable occupying the i-th position in 
the basis. 

The elementary transformation associated with the vector A, for which 
x,=a, (j¢/,) is, thus, defined as transformation (5.14) with 6 satisfying (5.19). 
The parameter 6, in inequality (5.19) is determined from (5,21). 

2. We now introduce the concept of elementary transformation for the 
second case. 

In this case, to vector A,, associated with the elementary transformation, 
there corresponds an extrabasis variable x,=6,. 

The set of elementary transformations of program X to program X(6) 
associated with the vector A, is given by 


x,—6, if p=, (5.22) 
Xp if wely, pos. 


The vector X(6) is a program of problem (5.1)-(5.3) if and only if 


Xs, + Ox;,, if p=s, i=1, 2,...,m, 
x, (6) = 


0<6<6,, 


where 86, is the least of the three numbers 
—X 


a 
oo) = min 1 ; 
Xyjco xij 
a = min Pao 78 
xy>o | ty 


Let 


_ Gs,, if %;;<9, = 
= B,, if x,,>0. (i=1, 2,...,m+1). (5.23) 


In our notations, we have the following compact formula for §,: 


i= min Yi Xie (5.24) 
xyxeo | Fs 
Siem 
Here, as before, %gi,,;=—1, 2nsi=*%/=%,.. For all nonnegative 6 not 


larger than 6, (as defined in (5.24)), the elementary transformation associa- 
ted with the vector A, for which x,= 6, (j¢/,) transforms the support program 
X to a program X(#) of problem (5.1)-(5.3). The proof of this proposition 
is identical with the proof of the analogous assertion for the first case, 
when X,=G,, and can, therefore, be omitted. 

In the following 6, will be called the elementary-transformation 
parameter. 

Thus, (5.14) and (5.15) for 0<6<6,, where 6, is computed, respectively, 
from (5.21) and (5.24), uniquely specify the elementary transformation as- 
sociated with any restraint vector A,, j@¢/,. 

Let us now trace the change of the linear form under the elementary 
transformation associated with the vector A,, j¢/,. 

Applying (5.14) (for the first case) and (5,22) (for the second case), 
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we obtain a 
(cy— 2&4 %i,) for x,=a,, 

L[X()J—L(X)= m 

0 (2 cu %1j—6)) for x,=f,. 


According to the definition of A, (see (5.6) and (5.7)) we obtain the following 
result 
L [X(6)J—L(X)=—6A,. (5.25) 


Formula (5.25) shows that the parameter A, can be interpreted as the eva- 
luation of the vector A, with respect to the program X with the given basis 


A,, A, eed »A, 


or, more briefly, the relative evaluation ofA, 

5-5. We now give the description of the general procedure of solution 
of the linear-programming problem with bilateral restraints. 

In this section problem (5.1)-(5.3) is assumed to be nondegenerate. In 
Chapter 2, 4-7, nondegeneracy was defined for the linear-programming 
problem in arbitrary form. It can easily be verified (see Exercise 5) that 
for a problem with bilateral restraints this definition can be given as follows. 

A support program of problem (5.1)-(5.3) is called nondegenerate, 
if all its basis components satisfy inequalities (5.4). 

In other words, a support program of a problem with bilateral restraints 
is considered nondegenerate if its m components satisfy restraints (5.4). 

As before, a nondegenerate problem is a problem whose support 
programs are all nondegenerate. To solve problem (5.1)-(5.3), we set out 
from some support program and the procedure is terminated after a finite 
number of homogenous iterations. 

We will describe one step of the procedure. Let X=(x,, x,,...,x,) bea 
support program of problem (5,1)-(5.3) with the basis A,, A,,...,4,. The 
program X is given or has been obtained in the preceding iterations. Each 
iteration can be divided into two stages. In the first stage, the program X 
is tested for optimality. To this end the relative evaluations A, of all the 
vectors A, not appearing in the basis (j¢/,) are calculated. Depending on the 
method of computing z,, the first or the second form of the optimality test 
is applied. 

The following cases arise. 

(a) If all the A, (see (5.7)) prove to be nonnegative, the program X is 
optimal, and the solution procedure is terminated. 

If the A, of some of the restraint vectors A;(j¢/,) are negative we consider 
the elementary transformations associated with each of these vectors. 

(b) Among these elementary transformations there is at least one whose 
6, is infinite. 

(c) For all the elementary transformations 4, is finite. 

We analyze each of these cases. 

From (5.21) and (5.24) we see that in case (a) there exists a vector A, 
with a negative evaluation A, such that 

@,,=-—oo for all x,,>0, 
6,,= oo forall x,<0, 


if x,=a,, and 
a,=—oo for all x,<0, 


= oo for all x,,>0, 
if t= By. Bs, th 
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Obviously the vector X(6) obtained under the elementary transformation 
associated with the vector A, is a program of problem (5.1)-(5.3) for any 
§>0. Therefore, (5.25) (for j=) shows that the linear form (5.1) is un- 
bounded in the set of feasible programs of the problem. The solution pro- 
cedure is terminated in this case when unsoivability of the problem is es- 
tablished. Observe that case (b) cannot arise if a, and 8, are finite. 

In case (c), we must proceed with the second stage of the iteration. In 
the second stage we construct a new support program for problem (5.1)- 
(5.3) which is closer to the optimum. The transition to the new program is 
effected by the elementary transformation of program X associated with 
any vector A, whose evaluation A, is negative. Let 


X’ = X (6,), 


where 6 is the parameter of the given elementary transformation. The 
components of the vector X’ are determined from (5.14) if x,=a,, and from 
(5.22) if x,= 8, The parameter 6, is computed from (5.21) or (5.24) depend- 
ing on the actual limit with which the component x, coincides. In all these 
relationships /= Rk. 

For the sake of convention we say that 6 is obtained on the i-th position 
of the basis (or on the vector A,,) if 


Xte—Vi 
——— a 
a for x,=4,, 


Here i=1, 2,..., mm+1. 

We recall that the (m+ 1)-th position of the basis is occupied by the vector 
A,=A, selected to be introduced into the basis of the new program. 

Assume that 6, is obtained on the r-th position of the basis. Then, ob- 
viously, the component x,,=.,,(6,) is equal to one of its limits (a, or B,). Thus, 
the components of the program X’ with the indices jély, /#k and j=s, are 
equal to their limit values. It can easily be verified that the restraint vec- 
tors corresponding to the remaining variables are linearly independent. 
Indeed, for r#m+1 this follows from Theorem 2.1 of this chapter. If, 
however, r=m+1 (this case is quite feasible), the system of vectors being 
considered is the basis of the support program X and is a fortiori linearly 
independent. 

We conclude that the new program X’ is a support program of the prob- 
lem. Fors,=& the basis of program X’ coincides with the basis of program 
X; for s,~#k the basis of X’ is obtained from the basis of X when the vector 
A, is replaced by the vector A,, The vector A, is introduced into the r-th 
position of the basis. 

It is worthy of note that since problem (5.1)—(5.3) is assumed nonde- 
generate, 6, can be obtained only on one position of the basis. Otherwise, 
the support program X’ would be degenerate. Hence, the index r is in this 
case uniquely specified. 

We see from (5.25) that when passing from support program X to support 
program X’ the linear form increases 


L(X’)—L(X) =—6,A,. (5.26) 


The elementary-transformation parameter 6, of a nondegenerate program 
is positive, Hence, 
P L(X')>L (2). 
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The second stage of the iteration, and the iteration as a whole, is ter- 
minated by the construction of a support program X’ with a higher value of 
the linear form. 

5-6. Successive iterations are carried out as long as the required solu- 
tion has not been obtained or unsolvability of the problem has not been 
established. Inthe nondegenerate case, each iteration increases the linear 
form (5.1). Inthe process of solving the problem we, therefore, continual- 
ly deal with different support programs. It can easily be shown that the 
problem has at most N=C,'-2"-" support programs. Indeed, the number of 
bases of problem (5.1)—(5.3) is at most Cy, and to each basis correspond at 
most 2*-* support programs (the extrabasis variables may equal either a, 
or B,). The process of solving a nondegenerate problem will, therefore, 
contain at most N iterations. In §6 we shall show that in the degenerate 
case, too, the simplex method yields a solution of the problem or establi- 
shes its unsolvability after a finite number of steps. 

In conclusion, we briefly repeat the sequence of operations involved in 
a single iteration. In the first stage of the iteration, the support program 
X is tested for optimality. The optimality test of a support program re- 
quires nonnegative A, for all j, The relative evaluations A, are expressed 
either in terms of the coefficients of the restraint vectors expressed in 
terms of the basis vectors or in terms of the components of the vector A. 

The first stage leads to one of the three possibilities (cases (a), (b), (c)). In 
case (a), support program X is the solution of the problem. Case (b) pro- 
ves thatthe linear form has no upper bound in the set of feasible programs of 
the problem, Inboth cases, the solution process is terminated. In case (c) we 
proceed with the second stage of the iteration. In this stage a new support 
program is constructed for which the value of the linear form is higher. 
The new program is construced with the aid of the elementary transforma- 
tion associated with some vector A, whose evaluation A, with respect to the 
previous program is negative. We thus see that bilateral restraints al- 
most do not affect the laboriousness of the computations of a single itera- 
tion in the simplex procedure. 


§ 6. Degeneracy 


6-1. Until now, in our discussion of the simplex method we invariably assumed the linear- programming 
problems to be nondegenerate. In the present section we extend the simplex method to the general case. 
The generalization is made for both nondegenerate and degenerate lineat-programming problems, Since the 
canonical form of the general line ar- programming problem is a particular case of a problem with bilateral 
restraints all the arguments (unless otherwise specified) refer to problem (5.1)-(5.3). 

The characteristic feature of a degenerate support program of problem (5.1)-(5.3) is that some of its 
basis components obtain their limiting values (a, or B,). Amy nondegenerate support program has a unique 
basis comprising the restraint vectors whose components satisfy inequalities (5.4). In the degenerate case, 
any given support program may have several bases. For example, if a support program of problem (1.1)~ 
(1.3) has v<m positive components, its basis is made up of v restraint vectors corresponding to these com- 
ponents and m—v restraint vectors associated with the zero components of the program. The only requirement 
needed to be fulfilled by the m—v vectors is that the resulting system of m restraint vectors be linearly 
independent, Obviously, the number of different bases of such a support program of problem (1.1)-(1.3) is 
at most C™", and in some problems this maximum estimate is actually obtained. Analogous considerations 
apply to problems with bilateral restraints (see Exercise 6). 

Below we examine all the stages of a single iteration of the simplex method stressing those points in 
which nondegneracy is assumed. 
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Thus, let 
X = (x, Xs eee, Xn) 


be a support program of problem (5.1)-(5.3) with the basis As,, As,, ..., As. The first stage of the itera- 
tion does not imply nondegeneracy of program X. In the second stage of the iteration the assumption of non- 
degeneracy is used twice: 

(1) when computing 6,; 

(2) when choosing the vector to be eliminated from the basis (or, equivalently, when determining the 
basis position in which a new vector is to be introduced). 

Let us consider each of these cases separately : 

1. When program X is degenerate, @ may equal zero if the component of the vector Ag introduced into 
the basis is equal to @y, which applies when for some i, lee igqm, 


Xjg=X, 0,» Xin > 0, 
or 


Xjo=z, =B., X <0 
(see (5.21). oy Bye Hie 


When x,=B, 6,=0 if for some ¢ 


Xjgp =X, =, Xin <9, 


ys? 
or 
Xjg =X, =f, » tp> 0 
(see (5.24)). 

If ¢,=0, the new support program X’ coincides with the previous support program X, The iteration only 
modifies the basis of this program. 

2. The support program X’ of the nondegenerate problem (5.1 )-(5.3) which is obtained when program 
X is improved is nondegenerate. 

Hence, @, is obtained on a single restraint vector A, (on a single basis position, r), The single vector 
A, is eliminated from the basis. In the general case, however, @, may be obtained on several vectors. Only 
one of these vectors should be eliminated from the basis. 

If in several successive iterations @=0, the solution procedure during these iterations involves trans- 
ferring to different bases of the same support program. Since the linear form obviously remains constant in 
the process, there is no justification of the assertion that different bases of the problem are involved. 

In Chapter 5, §9, weshall examine a linear- programming problem whose solution entails aso-calledcycle, 
i.e., periodic return, after several iterations, to a given basis. The solution procedure of such a problem 
will, obviously, not terminate. We shall show, however, that a slight improvement of the procedure will 
entirely eliminate the possibility of a cycle arising. It will be proved that if the vector to be eliminated from 
the basis is chosen, from among the several vectors on which @, is obtained, in a particular way, we in- 
variably deal with different bases of the problem and, therefore, after a finite number of iterations obtain an 
optimal program. 

6-2. Before giving the rule for vector elimination, we shall illustrate degeneracy geometrically. 

We shall apply the first geometrical interpretation. Consider a support program X=(x,, X,---:%,) Of 
a problem with bilateral restraints (5.1)-(5.3). To be specific, we assume that the first v components, 
vem, of this program satisfy condition (5.4). The support program X corresponds to the vertex X of the 
polyhedral restraint set. The vertex X is the intersection of m-+n—v hyperplanes the first m of which are 
defined by restraints (5.2) and the other a—vy by the equations 


xp=xyp=Vp  J=Vtl, 0, a, (6.1) 


where vy is either a, or B;. 

If v=m, the vertex X belongs to exactly w hyperplanes. If v<m, the number of hyperplanes inter- 
secting at the vertex is greater than the dimensionality a of the space. 

Geometrically, degeneracy of a support program implies that the number of iypetplanes intersecting at 
the corresponding vertex is greater than a, where a is the number of problem variables. 

Consider a basis of program X, 


A, A,, eee A, Aj,» Aj ones Aj,y! 


where v+l acj,acan, s=1, 2, ..., m—v. Geometrically, the choice of this basis indicates that the vertex 
corresponding to program X is considered as the intersection of exactly a independent hyperplanes, the first 
m of which are defined, as always, by restraints (5.2), and the remaining a—-m are hyperplanes of the form 
(6.1) with j#js, S=1, 2, ..., m—v. For v<m there may exist several such systems of hyperplanes and, 
consequently, several bases of program X. 
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Let A, be an arbitrary restraint vector not appearing in the given basis of program X. We eliminate the 
hyperplane x;=y, from the system of hyperplanes associated with the basis. The intersection of the remain- 
ing hyperplanes is a line passing through point X. We shall say that the ray issuing from the point x parallel 
to this line toward the polyhedral set of the problem corresponds to the vector A, Analogous definition 
for the canonical form of the linear-programming problem will be found in § 4. 

The elementary transformation of program X corresponding to the vector A, is geometrically equivalent 
to translation along this ray. While in the nondegenerate case the intersection of any such ray with the poly- 
hedral restraint set is, necessarily, a segment, this intersection in the degenerate case may contain only one 
point, X. Thus, any infinitesimal translation along the given ray gives rise to a point which does not belong to 
the domain of definition of the problem. 

Let the vector A, be chosen to be introduced into the basis in the first stage of an iteration. The linear 
form of the problem increases along the ray corresponding to this vector. If by translation along this ray we 
do not leave the polyhedral restraint set (6,>0), the second stage will give a new support program on which 
the value of the linear form is higher. If no such translation is 
possible (6,==0), we remain at the same vertex X. The second 
stage of the iteration will only modify the basis of program X. 
One of the hyperplanes x,=xg in the set of hyperplanes corres- 
ponding to the previous basis will be replaced by a hyperplane 
of the form (6.1) with j=j,, s=1, 2,...,m—v. If we do not 
specify which of these hyperplanes should in fact be introduced 
in the system corresponding to the new basis, we may arrive 
after several iterations to a system of hyperplanes which has al- 
ready been examined, i.e,, a cycle arises. 

To clarify the geometrical meaning of degeneracy, consider 
a linear-programming problem whose restraint polyhedron is a 
regular hexahedral pyramid (Figure 4.7) (the problem restraints 
are assumed to be defined by inequalities). 
The vertex S, of the restraint polyhedron correspond to a 
FIGURE 4.7 degenerate support program. Consider the six planes to which 
the faces intersecting at S, belong. Any three of these planes 
represent some basis of the support program defined by S,. The support program in question thus has C%==20 
different bases. 

For example, consider the basis corresponding to the faces S,S.S,, S3S,Sq, S,S,S,. The lines of inter- 
section of any two planes to which these faces belong have only one point in common with the restraint poly- 
hedron (the point S,). Therefore, any single iteration will invariably leave us at point S,, i.e., we shall 
pass from the given basis of the program to another basis of the same program. This also applies to the basis 
comprising the faces SSS, S,SoS5, SgSeS;- 

Any other three of the six planes in question obviously define at least one direction which has a whole 
segment in common with the restraint polyhedron. On the other hand, in any such threesome there are two 
planes whose line of intersection has only one point in common with the polyhedron. From any such basis 
we may, therefore, proceed, after one iteration, to a basis of a new support program, or to a different basis 
of the same program. 

6-3. We now consider the concept of degeneracy in terms of the second geometrical interpretation (see 
Chapter 2, §5, and Chapter 4, §4). Since the second geometrical interpretation has been described only 
for linear-programming problems in canonical form, we shall limit the discussion to problem (1.1)-{1.3). 

Let 


X= (Ky, Xp oer Xn) 


be a support program of problem (1.1)-(1.3). To be specific, we assume that this basis comprises the first m 


restraint vectors. Then, = - 
sati™= eee = X, == 0. 


Let the first v basis components of program X be positive, and the remaining components equal zero. 

For v<m the support program X is said to be degenerate, Program X corresponds to some point P on 
the intersection of line Q and cone K of the given problem. This point also belongs to the m-dimensional 
cone Ky spanned by the augmented basis vectors of program X. If v<m, the point P is on the boundary of 
cone Ky (more precisely, inside its v-dimensional face, which is a cone spanned by the first v augmented 
restraint vectors). 

Let the vector A, be chosen to be introduced into the basis. We have seen (§ 4) that the new support 
program corresponds to the highest point of intersection of the line Q with the cone Ky spanned by the vectors 
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A, A, ase ap Am Ay. if program X is nondegenerate (v=m), the intersection of line Q and cone Ky is a 
segment (the problem is assumed to be solvable). The highest point of intersection of Q and K, is therefore 
distinct from the lowest point P. If, however, program X is degenerate, the line Q may intersect the cone 
K, inone point (the point P), which is on the boundary of the cone Ky, i.e., one of the faces of the cone Ky. 
In this case the highest point of intersection coincides with the lowest point P, and we have the same 
support program X. 

For m==2 all the preceding geometrical considerations become clear, Consider a linear-programming prob- 
lem described geometrically inFigure 4,8, Thesupport program with basis A,, As is obviously degenerate, since 
line Q intersects the side OA, of the angle A,OA,. Intro- 
ducing the vector A, into the basis does not increase the 
linear form, since the line Q and the cone spanned by the 
vectors A,, A;, Ag intersect at one point, P. As regards 
the vector A;, its introduction into the basis increases the 
linear form (the intersection of the line Q and the cone 
spanned by the vectors Aj, As, A, is the segment PP’). 

This geometrical interpretation of degeneracy points to 
the way of avoiding the danger of a cycle. Degeneracy 
of a problem means that the line Q intersects at least one 
of the polyhedral cones spanned by at most m—I aug- 
mented restraint vectors. it is obvious, geometrically, 
that some small parallel translation of the line Q will 
eliminate all these intersections, i.e., will reduce the 
problem to a nondegenerate one in which no cycle may 
arise. Translation of the line Q is associated with modi- 
fication of the constraint vector B which uniquely de- 

FIGURE 4,8 fines the position of the line. A cycle can thus be avoided 
if the vector B is suitably modified. This idea forms 
the basis of the rule derived for determining the vector to be eliminated from the basis without obtaining 
a cycle. 

6-4. This rule, which is derived in what follows, was first obtained by Charnes /1117/ for the linear- 
programming problem in canonical form. 

Let s>0O and R,, Ry, ..., Re be a linearly independent system of m-dimensional vectors. Consider 
the following linear-programming problem: 

Maximize the linear form 


n 
»2 CX, (6.2) 
=1 
subject to the conditions 
n 
p> x;A;=B (e), (6.3) 
=1 
a,x <6, j=), 2, ooey M. (6.4) 
Here 
m 
B(e)=B+ D> e/R,. (6.5) 


Problem (6.2)-(6.4) differs from problem (5.1)=(5.3) only in the constraint vector, which is a combination 
of the vector B, the system R,, Ry, ..., Rg» and the number e. 

Each linear-programming problem (5.1 )=-(5.3) is thus associated with an entire set of problems (6.2)= 
(6.4). The representatives of this set will be called e-problems. 

Our following analysis will proceed from two propositions on the €-problems. 

Theorem 6.1. There exists an e,>0 such that the e-problem (6.2)-(6.4) for 0<e<e, is non- 
degenerate. 

Proof. Let A,, Ag, -.., Ag. be a linearly independent system of restraint vectors. The set of indices 
of these vectors is denoted by /. We associate with the system of vectors A, | €/, a set of a-dimensional 
vectors X (e) satisfying restraints (6.3) and such that x,(e) for i€J7 is equal either to a, of to B;. 

From (6.3) and (6.5) 


m 
jet =1 1¢/ 
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where y,;=x,(e) is either a, or By. Hence 


x, )=4—> vayt % e!F4;. (6.6) 
iti =z} 


Here 6y, Gi, ry are the coefficients of A,, when the vectors B, A, and R;, respectively, are expressed in 
terms of the vectors of system Ay, j € /. ‘Note that none of the polynomials x,(6) is constant, Indeed, 

the square matrix |] Fis lim is the product of the matrices (A,, A,, ... A,)7? ~* and |[rijlla =(Ry Rey «+++ Rm), 
each of which has a nonzero determinant. The determinant of the matrix Ti yllen is also nonzero and, con- 
sequently, none of the rows of this matrix can comprise zeros only. 

Thus, for any #, some of the ip fJ=1, 2, ..., m are nonzero, i.e., the polynomials x, (e) F const. 
Henee, the polynomials x, (e)—a,, and B, —x, (e) do not vanish identically. Any polynomial (which does 
not vanish identically) of degree not greater than m has at most m positive roots. 

Let the least of the positive roots of the polynomials x s,(@)—G,, and B,, —*;,(e) be mu (/, y,). If there 
are no positive roots, we take mNj=co. Let the least n;(/, ¥,), t=1, 2, ..., m be n(/, ¥,). The parameter 
n(/, ¥/ is defined by a system of linearly independent vectors A,, | €/, and by the number y,, j¢/. The 
n restraint vectors can be arranged into a finite number of different systems each comprising m vectors. For 
fixed J, there exist at most 2"-™ different sets of y,, i€/. Therefore, 


e,=minn(!, y), 


where the minimum taken over all the possible systems of linearly independent vectors A, j{ €/ and sets 
of yj, equal to a, or By / €/, is positive. 

Now let 0<e<e,. Consider a support program of the e-problem with the basis A,, ..., Az. Itis 
obvious that the program is nondegenerate. Indeed, if for some # the basis component 2, (€) is equal to a, 
or B,, one of the polynomials 


%,,(8)— G,, » B,, —*,, (e) 
has a root 6. 

On the other hand, according to the definition of e,, the positive roots of these polynomials cannot be 
smaller than @,>8. 

Thus, the basis components of any support program of the é€-problem for 0<e<e, lie strictly between 
their boundary values which, by definition, implies nondegeneracy of problem (6.2)-(6.4). This completes 
the proof. 

Theorem 6.2. There exists an e,>0 such that tf 


0<e<e, 


and X (e) = (x, (e),..., %, (e)) §s @ support program of the e-problem with the basis Ags Ags.--, Ag, the 
vector X=(x,,X%q, «++, Xn), where x,=x,(e) for | #5, t==1,2,...,m and the other m components are 
determined from equations (5.2), is a support program of problem (5.1)~(5.3). Moreover, if 

X (e) is an optimal program of the e-problem, program X is a solution of problem (5.1)-(5.3). 


Proef. Consider a system of m linearly independent restraint vectors Ay { €/=(S;, Sg, ..+ 4 Sm). Let the 


n-dimensional vector X satisfy restraints (5.2), Let, moreover, x;=y, for jel be equal to a, or B;. Then, 
from (6.6), 
4u=b —Svpip i=], 2, eee m. (6. 7) 
ie€! 


Let the least of the positive numbers X3;—Bs, and Q,,— Xz, be §;(/, vy). lf Oe, GX%s, GBe,, we take 
(Lyme. Let 
q, min 1 ; 
5 ( Yew min bv) 


The parameter & (J, y,) is specified by the system of vectors Ay, /€/, and by the set of numbers y,, /G/, 
each of which may assume at most two values (a, B,). Therefore, 


= eh ECL, ¥). 


as the least of a finite set of positive numbers is also positive. 

We now introduce a number r(/)} equal to the modulus of the greatest (in absolute value) element of 
the matrix || riz \le constructed of the coefficients of the vectors R,, R,,..., Rg expressed in terms of the 
vectors of the system A i JEl. Let 

ae r(f). 
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Obviously, r>0. We take 
ey=min( 1)>0, 


rm 


We shall show that for any e, O<e<e,, to each support program of the e-problem corresponds a support 
program of problem (5.1)-{5.3) having the same basis and the same extrabasis variables, 

Let Ag,, Ag...» Asy, be the basis of the support program X (e)=(x, (e),..., %,(e)) of the e-problem. The 
vector X=(X,, Xy, ...,%q) satisfies equalities (5.2), and x,=x,(e)—=yy, for i€!. The basis components of 
program X(e), which are defined according to (6.6) by the equalities 


m m 
xs (e)=b;—)) Ai; +>) e/ Fj Xe, +> rip, i=1, 2, ..., M, (6.8) 
se! f=1 j/=1 


satisfy the conditions 
Gs, Xs; (2) < Bs,. 
We shall verify that 


Gs, & Xs, = b;— 9) ¥/ 41; = Bs, ; b= 1,2) nym: (6.9) 
i€! 
Indeed, if for some 4=& the component xq is, e.g., tothe left of ay, we have 


m m m 
¥% (e)=xq+ e/g <an—E+ D) ery <5 —E+ ryv< 
j= fat j= 
<—a5,—§+ mre<ay,. 


The first and the second inequalities follow from the definition of § and r, respectively. The third and the 
fourth inequalities follow from the definition of e.>e. 
The above relationships yield the inequality 


Xs (€)<@q, 
which contradicts the assumption that 4x4, (e) is a component of a program of the e-problem. Thus, 


Xu dg, é=1, 2, ..., m™. 


The restraints in the right-hand side of (6.9) are analogously verified. 

The vector X thus satisfies restraint (5.2) and restraints (5.3) simultaneously and is, consequently, a 
program of problem (5.1)-(5.3). By definition, the components x, of X for /@/ are equal to their boundary 
values. Therefore X is a support ptogram of problem (5.1)-(5.3) with the basis Ay, Ag,, .... Asm: 

Let now X (2) be an optimal program of the e-problem. Let further O<e<e,. From Theorem 6.1, the 
e-problem is nondegenerate. Hence the vector A=(A,, Ag, .... Ag) determined from 


(A, A)=c, 7E/, (6, 10) 
satisfies the inequality 
(A, A) cp (6.11) 
if j@/ and x,(e)=x,;=a, or the inequality 
(A, A) oy, (6.11') 


if jf and x,(e)=x/=fy. 

Indeed, if (6.11) and (6.11") did not, apply, we could have increased, by a single simplex iteration, the 
value of the linear form of the nondegenerate e-problem. 

Since the support program X and X (e) have the same basis, relationships (6.10), (6.11), and (6.11') can 
be considered, with reference to program X, as the conditions of the second form of the optimality test. 
Program X therefore solves problem (5.1)-(5.3). 

The numbers e, entering the conditions of Theorem 6.2 can thus be defined as 


e,= min { ¢,, e,}>0. 
This completes the proof, 
6-5. These properties of the set of e-problems associated with problem (5.1)-(5.2) suggest the follow ing 
procedure for solving the problem without introducing, in any way, the requirement of nondegeneracy. The 
initial problem (5.1)-(5.3) is replaced by the corresponding e-problem with 0<e<e,, where 


e,=min {e,, e,} =e. 
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Let the set of feasible programs of this e-problem be nonempty. From Theorem 6.1 (0<e<e,), this e-problem 
is nondegenerate. The simplex method will, therefore, produce, after a finite number of iterations, its 
support solution. 

From Theorem 6.2 (0<e<&,) the basis of the solution is also the basis of the optimal program of problem 
(5.1)-(5.3). The extrabasis components of this program are equal to the corresponding components of the solu- 
tion of the e-problem. Hence, to obtain the basis components of the optimal program required it suffices to 
express the vector 

B— D2 xy (8) A; 
i¢r* 
in terms of the system A, j€J*. Here J* is the set of indices of restraint vectors which constitute the basis 
of the solution of the e-problem: 
x* (e) =(x, (2), x, (€), aos» x, (e)). 


The e-problem in question was assumed to be solvable. Note that if the process of solution of e-problem 
terminates in case (b), indicating its unsolvability, unsolvability of the principal problem (5.1)-(5.3) is also 
implied. The proof of this proposition follows from Exercise 7. To apply the e-procedure, the number @, 
need not be determined; only its existence need be established. 

The solution of the e-problem can be attempted with any sufficiently small e. To determine a solution 
of problem (5.1)-{5.3) it suffices to take @=0 in the optimal program of the e-problem. 

Since to each support program of the e-problem with 0<e<e, there corresponds a support program of 
problem (5.1)-(5.3) with the same basis (Theorem 6.2), the solution procedure of the e-problem involves exa- 
mination of support programs of problem (5.1)-(5.3). In a finite number of iterations the solution is obtained. 
We now establish the rules by which such examinations are made possible without considering the set of e- 
problems. 

Let X=(x,, Xq, .--. X,) be a support program of problem (5.1)-(5.3) with the basis Ag,, Ag, ... As. 

Let the vector X(e), where x;(e)=x, for i€ly, be a program of the e-problem for 0<e<2&,. 

If the simplex method is now applied to the e-problem, starting with the support program X (e), then in 
a finite number of iterations, having examined the programs X (e), X)(e), ..., XM (e), we obtain the 
solution X*(e). The same procedure can be carried out by examining the support program X (0) =X, ¢ = 
=1, 2, .... N of problem (5, 1)-(5.3) and hereby the solution X*(0)—X* is reached, 

We shall illustrate the above by transformation from program X to program X“), By assumption, the 
bases of programs X and X(e) and their extrabasis components are identical. The first stage of the iteration 
can, therefore, be realized starting with program X. In the analysis of the second stage, as in the case of 
problems with bilateral restraints, we shall distinguish between two possibilities. 

1. The component associated with the vector Ag chosen to be introduced into the new basis assumes its 
left boundary value ag. To determine the vector to be eliminated from the basis, we compute, from (5.21), 


Q(e)= min *o)—V, (6.12) 
Xue FO Zik 
igism+1 


Inserting for xj) (e)=%s,(e), i=1, ..., m+1 in(6.12) its expression from (6.8), we have 


m 
zit De! ris—Vi 
6 (¢)== min — =! (6.13) 
Xth FO Xik 
IigiQmt+! 
Here, as before, 7;s, ¢=wl, 2, ..., m are the coefficients of the vector Ry, j=1, 2, ..., m, expressed 
in terms of the system Ay, Agy «+, Aggy Constituting the basis of programs X and X(e). Note that ra41. j=, 
Jol, 2, ..., M, SINCE Xuy4so (C)—= Xp (C)—= Xp. 
Since the e-problem is nondegenerate, there exists a unique index r on which 6,(e) is obtained. The 
vector A, is introduced in the r-th position of the basis to replace the original vector Ag. 
Relationship (6.13) shows that the index rf, for sufficiently small 6, can be obtained in the following way. 
A sequence of sets of indices é are determined recurrently: 


E,> E,D... DEt. 
Let E, be the set of indices on which 


%— min ~#—¥ (6.14) 
Xb 0 Xik 
1S aQam+i 
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is obtained. If the set Eg contains more than one element, we introduce the set Egy, comprising the indices 
$€£q on which 


nin — (6.15) 
€ 


is obtained. Otherwise, the process is terminated. Thus, the sequence of sets Eg ends when a set Ey con- 
taining one element is obtained. It follows from (6.13) that, e>0 being sufficiently small, this sequence 
produces the index r on which @,(e) is obtained. Since the e-problem is nondegenerate, @,(€) is obtained 
on a unique index and therefore, having formed a sequence of at most m-+1 sets Eg, we obtain a set Ey 
containing only one element r, the index of the basis position into which the vector Ag, is to be introduced. 

2. The component associated with the vector Ag chosen to be introduced into the new basis assumes its 
right boundary value Bg. To determine the vector to be eliminated from the basis, we must compute (see 
(5.24)) 

()= min WH%e, (6, 16) 
Xin KO Xth 
Is emM+1 


Relationship (6.16) can be rewritten in the form 


m 
Vi-Xien— Dy ef liz 
6,(e)= min em. (6.17) 
Xi FO “ik 
1s sm +1 


Hence, as in the first case, it follows directly that the index 7 of the basis positioninto which the vector 
A, should be introduced can be determined as follows. We form a set E, of the indices é on which 


(<=: min Yir*ie (6.18) 
Xik FO ih 
isismt! 


is obtained. The sequence {Eq} is then recurrently constructed: 
E,D> E,>... > E;. 


If Eq is found to contain more than one element, the set Eg,, is defined as containing the indices é € Eg 
on which 


4 
max 22! (6.19) 
teEg *ih 


is obtained. If Eg contains one element, the sequence is terminated (Eqg=E£,). The sequence of sets By 
containing at most m-+1 sets terminates, necessarily, with the set Ey containing one element r, the index 
of the required basis position. 

The preceding rule for establishing the vector to be eliminated from the basis ensures that no cycle will 
arise when solving problem (5.1)-(5.3). This rule is, thus, always suitable for obtaining the solution of 
problem (5.1)~(5.3) in a finite number of simplex iterations. 

We should again emphasize that to realize the rule we must 

(1) isolate a linearly independent system of vectors Ry, Rs, ...,Rq such that for a sufficiently small 
e> 0 the vector X (e) corresponding to the initial program X is a program of the e-problem; 

(2) know the coefficients of the vectors R,, Ry, ..-, Rg as expressed in terms of any current basis. 

When describing the simplex algorithms we shall always indicate the most expedient (from the compu- 
tational point of view) system of vectors Ry, Rs, ..., Ra. 

6-6. The canonical form of the linear-programming problem is a particular case of a problem with bi- 
lateral restraints (@;=0, B;= 0). Therefore, for problem (1.1)-(1.3), the selection rule ‘for basis vector 
elimination is a particular case of the general rule. We now formulate this particular rule. 

The set E, of the indices ¢ on which 

6.= min ie 
Xik>0 *ik 
isi«qm 
is obtained is constructed. If E, contains one element r, the vector Ag is introduced into the r-th basis 
position. Otherwise, the set E, is constructed of the indices { € Ey on which 
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is obtained. If E, contains one element r, we know which vector Aé, should be eliminated from the basis. 
Otherwise, the set Ey is constructed, etc. Finally we obtain, necessarily, a set Ey, O— tem, containing 
one element, which specifies the vector to be eliminated from the basis. 


§ 7. Methods for construction of an initial program 


7-1. In our discussion in preceding sections we proceeded from some 
initial support program without specifying in any way how this support pro- 
gram is to be obtained. There are various classes of problems where the 
determination of an initial support program is quite trivial. 

Let, for example, the restraints of a linear-programming problem be 
given in the form 


n 
mAs <= B, x; = 0, 
=1 


and the components of the vector B be nonnegative. In this case the prob- 
lem is reduced to the canonical form by introducing m additional nonnega- 
tive variables x,,,,...,%,4,, and corresponding unit restraint vectors 
Aner es Agee’ 
—— 
Anyi=(0,...,0, 1, 0,...,0)F. 
se 


In the linear form of the problem, the coefficients of the additional variables 
are all zero. 

An initial support program of the problem in question with its restraints 
reduced to canonical form is, obviously, 


X= (0, 6.250, By 09 D)s 
en ae” 


The basis of this initial support program comprises the unit vectors A,,, (é= 
= 1, 1.4, mM). The components x, in the expression of A, in terms of the 
basis vectors A,,, i=1,...,m are equal to the corresponding components of 
the restraint vector A,. 

Since there is no need to solve systems of linear equations to determine 
the x, in the initial stage, the corresponding computations of the simplex 
procedure are substantially simplified. However, it must be kept in mind 
that if we take the additional vectors as the initial basis vectors we, usually, 
start with a poorer first approximation than in cases when the basis con- 
tains at the outset some vectors corresponding to the principal variables. 
As arule, there are more iterations required for computing the optimal 
program by this technique. Nevertheless, since modern computers are 
used, it is justified to introduce the additional variables into the basis. The 
increased number of computations, which may occur in this case, only in- 
volves repeated similar operations, which are quite simple (see Chapter 5, 
§§ 2, 5). 

There are other classes of problems for which determination of an ini- 
tial support program requires substantially less computations than solu- 
tion of a problem with a given initial program. Unfortunately, this is not 
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always so with the general linear-programming problem. In general, com- 
putation of an initial support program is no less tedious than determination 
of a solution when the initial support program is given. 

We now give some general methods for computation of initial support 
programs. In the following (Chapter 5, §8) we shall return to this problem 
and indicate several instances for which these methods can be simplified. 

7-2, Consider a linear-programming problem in canonical form: 

Maximize the linear form 


n 
L(X)= Dex, (7.1) 
iat 
subject to the conditions 
Day) be i=l, 2, ooo, M, (7.2) 
=1 
x,20, /=1, 2,...52 (7.3) 


Here we may take all 6;>0. Otherwise, all the signs should be reversed 
in both sides of the corresponding equality. 

Also consider the following auxiliary problem: 

Maximize the linear form 


fa 
L(X)=— Birnat (7.4) 
subject to the conditions 
La! 
Pacis t Fnsi= Po i=1, 2, 000g M, (7.5) 
=1 
x; 20, J=\1, 2,...,a-+m. (7.6) 


We solve the auxiliary problem by the simplex method. A feasible initial 
program of the auxiliary problem is made up of the components 


x,=0 for jaa, 
Xn4( =; for i=1, eB 


To the components x, of the initial program with j=a+1 correspond m dif- 
ferent unit restraint vectors. This indicates that the initial program chosen 
is a support program of the auxiliary problem. The linear form Z is 
bounded above in the set of feasible programs of the auxiliary problem 
(L<0). Therefore, by the simplex method the optimal support program of 
the problem will be obtained after a finite number of iterations. Two cases 
are possible: 

a) The optimal value 72* is zero. 

b) The optimal value Z* is negative. 

In case (a) the optimal program of the auxiliary problem is a support 
program of the initial problem (7.1)-(7.3). Indeed, when Z*=0, all x2,,=0 
(i=1, 2,...,m). Hence, the solution X* (its first na components) of the auxil- 
iary problem satisfies the restraints of the initial problem. The optimal 
program of the auxiliary problem thus coincides with some feasible program 
of problem (7.1)-(7.3). By construction, however, this program is a sup- 
port program. 

The basis of the initial support program of problem (7.1)—(7.3) obtained 
by this technique may contain not only restraint vectors corresponding to 
the principal variables, but also unit vectors corresponding to the additional 
variables. Naturally, to the unit vectors of the basis correspond zero 
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components of the program. An initial support program whose basis con- 
tains unit vectors corresponding to the additional variables is a degenerate 
support program of the problem. 

The rank of the matrix of coefficients of (7.2) is obviously m if the basis 
of the initial support program contains only restraint vectors corresponding 
to the principal variables of problem (7.1)—(7.3). 

If additional unit vectors appear in the basis of the initial support pro- 
gram, the rank of the matrix |le,|| may be less than m. The rank, r, na- 
turally cannot be less than the number of restraint vectors of problem (7.1)- 
(7.3) appearing in the basis of the initial support program. In Chapter 5, 

§ 8 we shall show how to complete this set of linearly independent restraint 
vectors A, to the maximum system of linearly independent vectors of the 


matrix 
A=(A,, A, eeey A,)= Il Qi, ll een 


To the new restraint vectors introduced into the basis correspond zero 
components of the program. If the rank, r, of the matrix |la,,|| is m, the 
basis obtained comprises m vectors. If the rank r<m, the basis of the 
initial support program also consists of less than r vectors. In this case, 
m—r restraints linearly dependent on other equations of system (7.2) can be 
eliminated during the construction of the initial support program thus re- 
ducing problem (7.1)—(7.3) to a similar linear-programming problem in az 
variables and r equality restraints. 

In case (b), when Z*<0, problem (7.1)—(7.3) has no feasible programs, 
i.e., the restraints of the initial problem are inconsistent. This can easily 
be demonstrated by reductio ad absurdum. Indeed, assume the problem to 
have at least one feasible program. Under this assumption, the auxiliary 
problem has a feasible program whose first 2 components coincide with 
the corresponding components of the initial program, and the remaining m 
components are all zero. The linear form vanishes on this program, which 
contradicts the assumption that the maximum value /*(X) is negative. 

An initial support program of any linear-programming problem can thus 
be computed by solving the auxiliary problem with an obvious initial program. 
If the problem restraints happen to be inconsistent, this inconsistency is 
detected during the computations. 

7-3. It is not always expedient to divide the solution of a linear-pro- 
gramming problem into two stages, i.e., computation of an initial support 
program and determination of an optimal program. Let us see how these 
two stages can be combined. Essentially, the solution of the initial linear- 
programming problem is replaced by the solution of an augmented problem 
with a larger set of support programs (one of which is always obvious) and 
having the same solutions, i.e., the same optimal programs, as the initial 
problem. 

Thus, together with the initial linear-programming problem, consider 
the following augmented problem: 

Maximize the linear form 


Rn m 
tll Bef 
where M> 0 is a sufficiently large number, and 
RNG 4B isicas tae) 
satisfies restraints (7.5)-(7.6). 
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We shall refer to the augmented problem as the M-problem. We shall 
prove three theorems necessary to justify this new approach. 


Theorem 7.1. If in the optimal program X* of the M-problem xn4;=0 
(imi,...,m), i.e., if X*=(xi,...,%0,0,...,0) the program X*=(x', x},...,%,) tS 
a Solution of the initial problem. 

Proof. If X* satisfies the restraints of the augmented problem, X®* satis- 
fies the restraints of the initial problem. Hence, X®* is a feasible program 
of the initial problem. 

Let-us establish optimality. Assume that a feasible program X=(x,, 
%,,++.,%,) Of problem (7.1)-—(7.3) exists such that 


L(X) > L (X°); 
X=(x,,..+,%,,0,-..,0) is a feasible program of the M-problem. Hence, 
Ly(X) = L (X) > L (X*) = Ly (X°). 


This equality contradicts the assumption that X* is a solution of the M- 
problem. 

Thus, if in an optimal program of the M-problem x*,,=0 fori=1, 2,...,m, 
the first 2 components of this program specify a solution of the initial prob- 
lem. This completes the proof. 

Theorem 7.2. There always exists a number M,>0, such that for any 
M> M, the existence of at least one support program of the initial problem 
implies 

Xo 4 =0 ({=1, 2,..., 18), 
for any support solution of the M-problem (if the latter is solvable). 
Proof. Let the proposition be false, i.e., assume that a support pro- 


gram X=(x,,...,*,) of the initial problem exists but that there is no M, such 
that for all M>M, x*,,=0(i=1, 2,...,m) for the support solution X* of the 
M-problem. 


We introduce numbers m and m as follows. Let m be the minimum of 


the sum Sx. on all the support programs of the M-problem for which this 
fel 


sum does not vanish. This minimum m>0 exists, since the number of 
support programs of the M-problem is finite, and is independent of M. Let 
m be the maximum of the linear form Dex, in the set of the support pro- 
grams of the M-problem. Obviously 


Let m => L(X). 
m=" 


and let us take M>M,. By assumption, at least one of the components 


x, (i=1,...,m) of the optimal program X* = (x1, .ee;Xnam) Of the M-program is 


positive. Therefore, from the definition of m and m, 


m 


n 
X44 SM, C)X, & Mm. 
Hence, ft cs 
Rn m = 
Ly (X*) = Paes*| —M 2 Fn < m—Mm, 


On the other hand, filling the program X of the initial problem with 
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zeros, we obtain a program X=(x,,...,%,, 0,...,0) of the M-problem. Here 


Ly(X)=L(X), Ly (X) Ly (X), 


since X* is a solution of the M-problem. 
Applying the last three relationships we obtain 


L(X) >m— Mn, 
or (since m>0) = 


This inequality contradicts the condition M>M,. This completes the proof. 
Theorem 7.3. There exists a number M, such that for M> M, solvability 
of the initial problem (7.1)-—(7.3) implies solvability of the associated M- 
problem. 
The proof is based on certain features of the duality theory presented 
in Chapter 3. Consider two linear-programming problems, dual with res- 
pect to the initial problem and the M-problem, respectively: 
1. Minimize the linear form 


m 
Pim (7.7) 
subject to the conditions 
m e 
D Fi S Cp jJ=1, y Seren (7.8) 
é=1 
2. Minimize linear form (7.7) subject to conditions (7.8) and 
¥,=— 4, i= 1, 2, 000, MM. (7.9) 
Let A,, Ay, +. Ay. be a system of linearly independent restraint vectors. 
We determine the vector Y,=(y,, ¥,.-+,¥e); from the equations 
m 
Pe tsi = Eyer J€f=U,) Jy oes Pe (7.10) 


Let y, equal the smallest components of the vector Y, and let 


M,= ares (— y)), 


where the maximum is taken over all the possible systems of m linearly 
independent restraint vectors. 

Assume now that M> M, and that problem (7.1)-—(7.3) is solvable. In 
this case, from the first duality theorem, the dual problem (7.7)—(7.8) is 
also solvable. 

Let Y=(y,, Ys.-++:¥,) be a support program of this problem. Its compo- 
nents satisfy equations (7.10) for some set /J=(/,, jy, ...,J_): 

From the definition of M, 

y2—M,>—M, — i=1, 2,...,m. 
Hence restraints (7.8), (7.9) are consistent and problem (7.7)-(7.9), differ- 
ing from the solvable problem (7.7)-(7.8) in the additional restraints (7.9), 
is also solvable. 

Applying the first duality theorem, we conclude that the M-problem 
(dual with respect to problem (7.7)-(7.9)) is solvable. This completes the 
proof, 
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The preceding results indicate a procedure for solving a linear-program- 
ming problem whose initial support program is not available to begin with. 
Rather than solve the initial problem (7.1)—-(7.3), we apply the simplex 
method to the corresponding M-problem where 


M>max(M,, M,). (7.11) 


An initial support program of the M-problem is, obviously, 
X,=(0, 0, ..., 0, b,, &, ..., Oy). 
Rea, core 

The basis constructed of unit vectors for the artificial variables x,,,, 
i=1, ..., m is generally calledan artificial basis. 

The simplex method starting from a support program with an artificial 
basis leads, after a finite number of iterations, either to case (a) (optimal 
program) or to case (b) (unsolvable problem), 

Let now X*=(x,, ..., X,+m) be the last support program constructed during 
the solution of the M-problem. Ifcase(a) applies, X* is a solution of the 
M-problem. Here two possibilities must be considered: 

(i) Xpo1=...=Xn4m~=0; then from Theorem 7.1 X*=(x;, x1, .»-, %n) is an 
optimal program of the initial problem (7.1)-—(7.3); 

(ii) > x;>0; from Theorem 7.2 and from condition (7.11) we see that 


f=n+ 
problem (7.1)-(7.3) has no feasible programs, i.e., it is unsolvable. 


If the process of solution of the M-problem terminates in case (b), then 
according to Theorem 7.3 and inequality (7.11) problem (7.1)—(7.3) is a 
priori unsolvable. Observe that inthis case unsolvability of the initial prob- 
lem may be due either to unboundedness of the linear form (7.1) in the set 
of feasible programs of theproblem or to the fact that this set is empty. 
Construction of the corresponding examples is left to the reader (see 
Exercise 9). 

Thus, the simplex method applied to the M-problem with M satisfying 
condition (7.11) gives, after a finite number of steps, a solution of the initial 
problem, or else proves its unsolvability. Here it is not required to know 
in advance an initial support program of the problem in question, (7.1)—(7.3). 
The above method of solving linear-programming problems is generally 
called the M-method. 

We emphasize that to solve problems by the M-method there is no need 
to compute M, = max(M,, M,). When the evaluations A, are determined while 
solving the M-problem, M should be taken greater than any other number 
used in comparison. If the solution of the M-problem yields an optimal 
program with x,,,;=0 fori=1,2,...,m, the first m components of the pro- 
gram specify a solution of the initial problem. If at least one of the com- 
ponents x,,; of the optimal program is positive for any sufficiently large mM, 
the initial program is unsolvable. Unsolvability of the M-problem implies 
unsolvability of the initial problem. 

An example illustrating the application of the M-method will be given in 
Chapter 5, §8. There we give also some computational procedures asso- 
ciated with the realization of the M-method. 

The arguments presented in this section are with reference to linear- 
programming problems in canonical form. All the methods discussed here 
apply also to problems with bilateral restraints. In this case the auxiliary 
and the augmented problems should, obviously, be solved following the 
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rules given in §5. Observe that additional (artificial) variables of the 
auxiliary (augmented) problems are only bounded below (a,=0, B, = co). 


§ 8. Theoretical applications of the simplex method 


The simplex method described in detail in the preceding sections of this chapter should not be considered 
as merely a computational procedure for solving concrete linear-programming problems, It is also most 
useful if we want to obtain various qualitative results of the linear- programming theory. In this section we 
shall illustrate the possibilities of the simplex method in application to the proof of several important theo- 
rems of linear programming, which have been proved in Chapters 2 and 3 on the basis of different considera- 
tions, All the results are stated and proved with reference to the canonical form of linear-programming 
problems (problem (1.1)-(1.3)). The rank of the matrix A=(A,, A;,-.., A,) is assumed to be am. 


Theorem 8.1. (Existence of support program.) If the set of feasible programs of problem 
(1.1)-(1.3) is nonempty, it has at least one support program. 

Proof. Without loss of generality we may take all the components of the constraint vector Bum(,, 
b,, ..., b,)7 to be nonnegative. We associate with problem (1.1)-(1.3) the following auxiliary linear- 
programming problem: 

Maximize the linear form 


m 
L=— SD igs; (8.1) 
x at 
subject to the conditions 
: (8.2) 
p2 Bk tXneieb, tet, 2 ..., 
=A 
xj 0, j=I, 2, eesg n-+m, (8. 3) 


Obviously, the (m-+-n)-dimensional vector 
X,=(0, 0, eae 0, b,, b;, easy b,) 


is a support program of problem (8,1)-(8.3). Therefore, when solving the auxiliary problem by the simplex 
method we may adopt it as the initial program. If in the degenerate case we apply the rule discussed in § 6 
(here the vectors R,, R,, ..., Re can be taken as the unit vectors @,,@,...,@ comprising the basis of 
program X,), the solution of problem (8.1)-(8. 3) is obtained by the simplex method in a finite number of 
iterations, 

The linear form (8. 1) is obviously limited by 0 in the set of feasible programs of the auxiliary problem. 
Hence, the solution cannot terminate in case (b) (see § 2) and will necessarily lead to a support solution of 
the problem (case (a)). Let this solution be 


X= (x, 2, 0, el, el, 
By assumption, problem (1,1)-(1.3) has a certain program 


X = (x;, Xa eats Xn): 


The vector 
X = (x,, Xa eoesg Zn 0, 0, e@eoeg 0) 


is therefore a program of problem (8.1)-(8.3), This program is, obviously, associated with the zero value 
of the linear form (8.1) and, consequently, is a solution of the auxiliary problem, Then X, also makes 
(8.1) vanish, i.e. , 


a+m 
x$") 220, 
fant+1 


Hence, since the components x(t) are nonnegative, we obtain 


x0 for jem+l, ..., m+n, 
so that the vector 


x; — (x), 0), oes xi) 


is a program of problem (1, 1)-(1.3). 
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By construction X, is a support program of problem (8.1)-(8.3). Hence, the system of vectors A jy Cor- 
responding to x)>0 (j=1,2, ..., a) is linearly independent, X, is therefore a support program of problem 
(1. 1)-(1.3), This completes the proof. 

Theorem 8.2. (Existence of support solution.) If a linear-programming problem (1.1)-(1.3) 
is solvable, there is a support solution among its solutions. 

Proof. Since the set of feasible programs of problem (1.1)-(1.3) is nonempty, the problem, according 
to the Theorem 8.1, has a support program 


X=(X,, Xqy coer Xq)- 


Taking X as the initial program, we apply the simplex procedure. 

Since the problem is solvable, a finite number of iterations will produce the support solution, This 
completes the proof. 

Theorem 8.3. (Solvability of the linear-programming problem.) The linear- programming 
problem is solvable if and only if 

(i) the set of its feasible programs ts nonempty, 

(ii) the linear form of the problem is bounded in this set. 

Proof. Necessity is obvious. We shall prove sufficiency. 

From (i) and Theorem 8.1 the problem has a support program. As in the proof of Theorem 8,2, we take 
this program as the initial program and apply the simplex procedure. The solution should obviously terminate 
in case (a) (case (b) is ruled out by condition (ii)). Hence, the program constructed in the one but last itera- 
tion is a solution of the problem. This completes the proof. 

in conclusion we offer a simple proof of the first duality theorem. First let us recall the statement of 
the problem dual with respect to problem (1.1)-(1.3): 

Minimize the linear form 


p2 biys (8. 4) 


subject to the conditions 


m 


x4 Ajj Cp j=1, 2, coo, ft (8.5) 


First duality theorem. Solvability of problem (1.1)-(1.3) implies solvability of the dual prob- 
lem (8.4)-(8.5). Here the maxtmum of linear form (1.1) subject to conditions (1.2), (1.3) cotn- 
cides with the minimum of linear form (8.4) subject to conditions (8.5). 

Proof. Taking some support program of problem (1.1)-(1.3) as the initial program we. apply the simplex 
procedure. 

Since the problem is solvable, the solution process, in a finite number of iterations, yields the support 
optimal program 

Ds ay re =) 


which satisfies the requirements of the optimality test (case(a)). Thisindicates that the vector A=(A,, Ag, ..- 
, Am) determined from the system of equations 


3 


D> tipi=ep Ely, (8.6) 


tl 
i 


has the property 


5 aifhi = Sey, Ely (8.7) 


ha 


(here the second form of the test is used; the vectors A;, j€/y constitute the basis of program X). Rela- 
tionships (8.6) and (8.7) show that the vector A satisfies restraints (8.5), i.e., is a program of problem 
(8. 4)-(8. 5). 

We shall show that 


p? ct;= DD) OM. (8.8) 
=1 


(=1 
Indeed 
n m n m 
Vr= Y= Dy Daiki= De Dy aijhi= > hi >» aipty= Dy didi. 
j=1 lely lely f=1 j=1 =1 


The first and the third equalities hold since x,=0 for j €7 y-- The second and the fifth equalities follow, 
respectively, from (8.6) and (1.2). The fourth equality is simply the result of changing the order of summation. 
Now let Yom(y,, Ya. ++, Ye) be a feasible program of problem (8. 4)-(8.5). Then 


" n m m n m 
2s PSY) D8 = Dr DY apt D vibe 
f=1 j=1 i=1 f=1 fei é=1 
The first inequality here is a consequence of restraints (8.5), which vector Y satisfies. The other relation- 
ships hold for the same reasons as the fourth and the fifth in the previous chain of equalities, Thus, 


n m 
> ory 9:91 (8. 9) 


ji 


where Y is a feasible program of problem (8, 4)-(8. 5). 

Relationships (8.8) and (8,9) show that A is an optimal program of problem (8.4)-(8.5). The dual 
problem of problem (1.1)-(1.3) is solvable, and the optimal programs of the two problems are related by 
equality (8.8). This completes the proof. 

The preceding examples of proofs of several important theorems of linear programming show that the 
simplex procedure is a highly effective tool in qualitative investigations in the linear-programming theory. 
It must be emphasized that the application of this procedure always leads to constructive proof which elicits 
not only the fact, but also the feasible computational approaches, 


EXERCISES TO CHAPTER 4 


1, Prove the necessity of the optimality test (in the first or the second form) for the case of nondegenerate 
support program. 

2. Give an example of a problem in which the requirements of the optimality test are not necessary. 

3, Show that the linear form of problem (1.1)-—(1.3) with a nonempty set of feasible programs is unbounded 
if and only if the positive semiaxis Ou,,,;, belongs to the cone spanned by the augmented restraint vectors. 

4. Show that the concept of a support program introduced in § 5 for a problem with bilateral restraints 
corresponds to the concept of a support program for a linear- programming problem in arbitrary form, given 
in Chapter 2, §4, 

5. Prove that the concept of nondegeneracy introduced in § 5 for a problem with bilateral restraints 
corresponds to the general concept of nondegeneracy given in Chapter 2, § 4. 

6. Prove that 

(a) any system of m linearly independent restraint vectors of problem (5. 1)-(5.3) may serve as the basis 
of at most 2"~® support programs of this problem; 

(b) a support program of problem (5, 1)-(5.3) whose degree of degeneracy is p has at most CF oes +p 
different bases. 

Give examples in which the maximum estimates are attained. (The degree of degeneracy of a support 
program is p=:m—v¥v, where v is the number of basis components different from the bounds values of the 
corresponding variables.) 

1, Prove that two problems which have feasible programs and differ only in the constraint vectors are 
simultaneously solvable or unsolvable, 

8. Prove that a cycle is impossible in a problem whose support programs all have degree of degeneracy 
not exceeding 1, 

9, Show by examples that when the linear- programming problem is solved by the M-method, case (b) 
may be due either to unboundedness of the linear form or to inconsistency of the problem restraints. 

10, Prove with the aid of the simplex procedure that nondegeneracy of the linear- programming problem 
implies unique solvability of the dual problem, 
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Chapter 5 
THE SIMPLEX COMPUTATIONAL PROCEDURE 


In this chapter we describe the computational procedures for solving 
linear-programming problems by the simplex method. This will be done 
in the following order. In §1 relationships between the parameters of sup- 
port programs related by elementary transformations are established. 
Recurrence formulas for the parameters of successive iterations are de- 
rived. In §§ 2-6 the computational procedures associated with the two al- 
gorithms of the simplex method are given in detail. All computational pro- 
cedures are illustrated by numerical examples. 

The simplex method has been perfected to a higher degree than other 
procedures and the literature deals fairly extensively with the subject. In 
different sources we find different approaches to the method and analysis 
of the corresponding algorithms. In §4, which deals with the vector and 
the coordinate form of the method, the equivalence of the different ap- 
proaches to the simplex method is established, and the meaning of the 
transformations in simplex algorithms is elucidated. 

In § 7 the specific features of solution of linear-programming problems 
with bilaterally restrained variables are dealt with. In § 8 the sequence of 
computations leading to the determination of an initial support program is 
described. The last section, §9, deals with a relatively special question. 
The possibility of cycling in degenerate problems is studied and an example 
illustrating this phenomenon is constructed. 


§ 1. Relationship between the parameters of successive 
iterations 


1-1. The simplex method involves a sequence of successive elementary 
transformations of a given support program of the problem to another sup- 
port program which is closer to the optimum. In order to test the support 
program for optimality and improve it in each successive step, we compute, 
in each iteration, various parameters which help characterize and evaluate 
the program. 

In the preceding chapter we obtained systems of equations relating the 
simplex parameters with the problem restraints (the restraint vectors, the 
constraint vector, and the linear-form coefficients). As has been indicated, 
the solution of these systems of equations constitutes the most tedious 
stage of computations. However, the specific properties of elementary 
transformations make it possible to confine the solution of systems of equa- 
tions to the first stage, in which the initial data are prepared for the first 
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iteration. Applying the techniques of Chapter 4, §7, we distinguish be- 
tween the determination of the initial support program and all its charac- 
terizing parameters, onthe one hand, andthe solution of systems of equa- 
tions, onthe other. It is far more tedious to determine the parameters 

of the initial support program than to compute the parameters of all the 
subsequent iterations, since the latter are related by very simple recurrence 
formulas. 

The relationships between the simplex parameters for different algo- 
rithms are established according to the same principle. Below we prove a 
fairly general theorem from which we derive, as corollaries, all the re- 
currence formulas needed for solving the problem according to the various 
simplex computational procedures. 

Consider two systems of linearly independent vectors {Pi},-, and {P,}ier. 
The sets of indices / and / differ in only one element. The set / is ob- 
tained from the set / by substituting the index & for the index r, so that 


(eh = Le del: ae) 


Let ¢,, and (1; be the coefficients appearing when an arbitrary system of 
vectors Q, is expressed in terms of the two systems of linearly independent 
vectors {Pi}:¢; and {Pisien i.e., 


Q= 2buyPo (1.2) 
= p> bi /Py. (1.3) 


Let the coefficients yn, appearing when the vector P, is expressed in 
terms of the system {P;}:e; be known: 


fo, Vin (1.4) 

Here n,,#0, since the two systems {P}:¢, and {P,};e,, comprise linearly in- 
dependent vectors and differ only in the vector P 7 (Py). 

We shall now establish a relationship between {(;;, on the one hand, and 
Cij and n, on the other. 

We have the following proposition. 

Theorem 1.1. The coefficients ¢,, and (1; in the expansion of arbitrary 
vectors Q, in terms of the systems (P,:ej1 and {Pi}ier, which satisfy rela- 
tionships (1. 1) and (1.4), are related by the formulas 


by — 2 Ni for ik, 


ty= 1.5 
pe for i=k. ( ) 


Proof. From equality (1.4) we have 
P,= 2, NinPit NeaPe 
ier 
By assumption n,#0. Therefore, applying (1.1) and (1.2), we obtain 


A= y ca toe Zo) a> (Gu —E nu) Pt P,. 


tel 
ier igh 


The system {P},,¢, is linearly independent. Therefore, the coefficients of 


#r 
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the vector Q, expressed in terms of the vectors of this system are uniquely 
defined. Comparing the last relationship with (1.3), we obtain (1.5). This 
completes the proof. 

The vector P, in the system {P,};~,, occupies the same position as the 
vector P, inthe system {P,;},., We have already seen that in some cases 
the indices of the components are most suitably related with the positions 
of the corresponding vectors in the system, rather than with the actual in- 
dices of thesevectors. In view of this we rewrite the recurrence formulas 
(1.5) in the form 


—f/n. for i 
ae Gry we ca Ff, 


bd for l=r. (1.6) 
"rk 


Here we assume that P_ occupies the r-th position in the system {P;}r:e:. 

1-2. We shall use (1.6) to derive recurrence formulas for the para- 
meters of successive iterations in the first simplex algorithm. 

For the first algorithm, associated with the first form of the optimality 
test of support programs, in each iteration the following parameters must 
be computed: 

(a) x,,, the coefficients in the expansion of any restraint vector A, (/=1, 
2, ..., 2) in terms of the basis vectors; 

(b) x,, the basis components of the support program (/= 1, 2, ..., m); 

(c) A,, the evaluations of the restraint vectors A; with respect to the 
basis in question (j=1, 2, ..., a); 

(d) L(X), the value of the linear form on program X. 


Let _ 
V=— gays 
where e,,,=(0, ...,0, 1) is the (w+1)-dimensional unit vector. Let 
{Pihier=lApier,» V}, (1.7) 
{Pier ={(Apier,.» V}. (1.8) 


The first system consists of the m augmented restraint vectors, specified 
by the basis of the support program 4X, and the (m+ 1)-dimensional vector 
V. The second system consists of the augmented vectors, corresponding 
to the support program X’, and the vector V. The support program X’ is 
obtained from the support program X by the elementary transformation 
associated with the vector A,. The two systems differ in one vector. The 
vector A, in the second system occupies the r-th position in the basis. In 
the first system the vector A, is in this position. The vectors of both the 
first and the second systems are, obviously, linearly independent. 

We take as the vector Q,; any augmented restraint vector Ap j=0, 1, 2, 
++, a. (The augmented vector A, is defined as the (m+ 1)-dimensional 
vector whose first m components coincide with the corresponding compo- 
nents of the constraint vector, and whose (m+1)-th component ¢, is zero.) 

The expansion of the vector A, in terms of the vectors of the first sys- 
tem can be written as follows: 


4,;= 2 biAn tomas Vs j=0, 1, «00, (1.9) 


Analogously, the expansion of the vector A, (the equivalent of the vector P, 
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in Theorem 1.1) is: 
L. | 
A,= Py TAs + Nass, aVe (1.10) 


The first m components of the augmented vectors specify the restraint vec- 
tors and are related by 


A,= B xyAu j=0, 1, 2, ..., 2, 
and, correspondingly, 
m 
A,= Pz Xia 
Since the first m components of the vector V are all zero, we have 
Ci=Xip m1, 2, 1.55%, Jo20, 1, o00, 2, 


Nie = iz i= 1, 2, ooey MM. 
Analogously, 


Cp = X14), i=1, 2, @eey m, J=9, 1, oeey f. 


To compute a4,,, and Nas,,, for j=0, 1, ..., awe project both sides of equa- 
lities (1.9) and (1.10) onto the e,,,-axis or, equivalently, compute the 
(m+ 1)-th components of the vectors in the left- and the right-hand sides of 


each of these equalities: i 


m 
Cp = 2 X inl 5; — Nasi, be 


Comparing these formulas with (2.1) of the previous chapter and since 
c,=0, we obtain 


Cet, 7a Ay J=1, 2, ..., 4, Na+, na Ay 
Cat, e=A,=L(X). 


Similarly, ; aE ee P 
m4+i, [= Of, = 1, 2, «05, A 


are = A, =L (X’). 
The following notations are obvious: 
V=A,_ |, Aj=Xm4,,p J=1,2,...,0% A =L(X)=Xq4,,9) 


and, correspondingly, 


Ap= mss, fy Jl, 2, cee Mm AGL (X) = Xmas 
In these notations, (1.9) and (1.10) become 
m+ 
A,= > 7A; jJ=9, 1, oeey A, 
ae ie (1.11) 
m+1 
A,= > X ipAsy. 


From the definition of the elementary transformation we have n,=x,,#~0 
(the index r gives the position of the basis on which the ratio “2, with 
X, > 0, reaches its minimum). 

All the requirements of Theorem 1.1 are thus satisfied. Applying the 
prop >sitions of this theorem to the expansion coefficients of the augmented 
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restraint vectors in terms of the vectors of systems (1.7) and (1.8), we ob- 
tain the following result: 
The parameters of two successive iterations are related by the recur- 
rence formulas 
x1] = wd for i=r, (1.12) 


i=1, 2, ..., m+1, j=0, 1, 2, ..., 2. 


Formulas (1.12) are quite sufficient for proceeding from a given step of the 
simplex method to the successive one. Recurrence formulas (1.12) make it 
possible to compute the following parameters of each iteration (provided 
the parameters of the preceding approximation are available): 

(a) the expansion coefficients of all the restraint vectors expressed in 


terms of the basis vectors (i=1, 2, ..., m; j=1, 2, ..., 2); 
(b) the basis components of the support program (i=1,2,...,m; j=0); 
(c) the relative evaluations of the restraint vectors (¢(=m+1; j=1, 2, ..., 2); 


(d) the value of the linear form (i=m+1; /=0). 

Formulas (1.12) constitute the basis of the first simplex algorithm. 

1-3, We now use (1.6) to derive recurrence formulas for the parameters 
of the successive iterations in the second simplex algorithm. 

The second algorithm is based on the second optimality test for support 
programs in which the evaluations of the restraint vector, A,, are com- 
puted in each iteration with the aid of the vector A=(A,,...,4,). The para- 
meters 4,, in turn, satisfy the equations 


iaidi=ep JE ly. (1.13) 


Computation of 4; involves finding the inverse of the matrix Ay. 

To solve problems by the second algorithm, we must compute the fol- 
lowing parameters in each iteration: 

(a) e,,, the elements of the inverse matrix Ax’; 

(b) 4,, evaluations of problem restraints with respect to the basis of the 
Support program X; 

(c) x,;,, the basis components of the support program; 

(d) L(X), the value of the linear form on program X. 

Observe that it is convenient to take the parameters e,,-the elements of 
the matrix Ax'- as the coefficients of the m-dimensional unit vectors 


SS 
v-(2 oes 0, 1, 0, eens 7 
Ne 
/ 


expanded in terms of basis vectors. 
Let, as before, systems {P},~, and {P,};¢,be defined by (1.7) and (1.8). 
The vectors Q, here are taken as the (m+ 1)-dimensional unit vectors 


e;= 0) aide Oe Le Oe 
ed 


, 


and 
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The expansion of the vector e, in terms of the vectors of the first system 
has the form 


€)= 2 by Aunt laa iV j=0, 1, 2, ..., m. (1.14) 


The expansion of the m-dimensional vectors e, in terms of the basis vectors 
(A,,)by definition has the form 


La | 
ey = BeAr jJ=9, 1, 2, cary m8. 
Hence 
Cis esp i=1, 2, + ade) mM, J=9, 1, eee m 


(e,,=x;, are the basis components of the support program). Similarly, 


by = ei), é==1, 2, ..., mM} j=0, | ree 


To compute e,,,,,/=1, 2, ..., a), we project both sides of equality (1.13) 
onto the vector @,,.: 


m 
o= 2 Cj, — Omar, ih 
Multiplying both sides of the last equality by a;, and summing over / from 


1 to m, and since a,, ande,, (i=1, ..., m; j=1, ..., m)are the elements of 
mutually inverse matrices, we have 


m 
O= Cy — 2 ainbmer, p f=1, 2, ..., m 


Replacing the index / by i, and since the indices s,(¢=1, 2, ..., m) constitute 
the set ly, we have 


Py Oi cms imep JE ly. 


The vector {&n4;,;},is, thus, the solution of system (1.13). Hence 
A js JH 2 peng Ms (1.15) 


Cia — 
Taking /=0 in (1.14) and rewriting this equality for the (m+ 1}th compo- 
nents of the vectors A,and V, we have 
0= Bo icu— Ena, 0° 


Cais, = 2 Xgl L (X)=A,. 


The following notations are obvious: 
VeAg i Aylmer, pp JEL eee mi 
Ag=L(X)=eCnsios (1.16) 
and, correspondingly, 
Myemas, fp SHV, coe, mi WoL (X’)=lmaro- 
We have, moreover, assumed that 


Xig=Cjp, f= 1, 2, ..., m, (1.17) 


173 


and, correspondingly, a 
Xio= Clo. 


In the new notations, formula (1.14) becomes 
e;= > @ As j=), I; 2, cooy M. (1.18) 
é=1 


The expansion of the vectors e=Q, in terms of the vectors of system (1.8) 
can be given by an analogous formula: 


e= eA, f=, 1, 2, ..., m. (1.19) 


Let J (84. Sy. sex SA ess, SQ); I =(5;, Ss, pisces Spy ea eySm)i 
j s, for is¢r, 
sim k for i=r. 
The expansion of the vector A,=P, in terms of the vectors of the first 
system is written, from (1.11), in the form 


m+. 
t=] 


Now, applying Theorem 1.1 to the expansion coefficients of the vectors e, 
in the vectors of the systems (1.7) and(1.8) and applying formulas (1.18)-—(1.20), 
we obtain the following recurrence formulas for the parameters of two suc- 


cessive iterations: a a ee 
a ij ae ’ (1.21) 
—£ for it=r, 
Xrk 
i=1, 2, ..., m+1, j=0, 1, 2, ..., m. 

Formulas (1.21) make it possible to compute the following parameters of 
each iteration, if the analogous parameters of the preceding approximation 
are known: 

(a) the elements e,, of the inverse matrix Ax'of the basis vectors (i=1, 

6, cg A JH 1, 2e.o 55m): 

(b) the basis components of the support program x,;,=e;,(i=1,..., m; /=0); 

(c) the relative evaluations of the problem restraints 1,=e,,,,, (i=m+1; 
= 1,2, 2025. m); 

(d) the value of the linear form L(X)=e,,,,,(/=m+1; /=0). 

The recurrence formulas (1.21) constitute the basis of the second sim- 
plex algorithm. 


j 


§ 2. The first simplex algorithm 


2-1, The theoretical principles of the first algorithm were discussed 
in Chapter 4, §2. Here we describe the sequence of computations to be 
followed when this algorithm is applied to the solution of linear-program- 
ming problems in canonical form. 

We have already seen that the simplex method, proceeding from some 
initial support program and gradually improving it, produces, after a 
finite number of iterations, an optimal program or establishes unsolvability 
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of the problem. Each iteration involves corresponding transition from one 
tableau of the algorithm to the next. 

Let us introduce some terms which will simplify the presentation. The 
concept row-row product or row-column product will refer to multiplica- 
tion of the corresponding rows or a row and a column in the tableau by a 
scalar. We shall say that a row (column) is divided or multiplied by a num- 
ber, if each element of the row (column) is divided or multiplied by this 
number. When saying that two rows (or two columns) are added or that one 
row is subtracted from the other, we shall imply that the operations speci- 
fied are carried out on the corresponding elements of the rows or the 
columns. 

Each iteration in the simplex procedure consists of two stages. In the 
first stage, the support program in question is tested for optimality. The 
second stage is carried out if the program considered is not optimal and 
the problem has not been shown to be unsolvable. In the second stage the 
elementary transformation, which produces a new support program with a 
higher value of the linear form, and the vector to be introduced into and 
that to be eliminated from the basis are determined. Then the basis com- 
ponents of the new Support program and all the parameters needed for pro- 
ceeding withthe solution are computed. 

Any vector whose evaluation with respect to the preceding basis is ne- 
gative can be introduced into the new basis. The increase of linear form 
will be maximum in each iteration if we introduce into the basis the vector 
A, for which OA, is minimum. Experience shows that this, as arule, also 
reduces the number of iterations needed to yield a solution. However, 
establishing for which vector the product 6A, is minimum is a fairly com- 
plicated procedure. First it is necessary to calculate 


0) — min Ze 


xy>o *is 
for all j such that A,<0. Then the products 6,2A,;must be compared. The 
index / corresponding to the minimum product specifies the vector to be 
introduced into the basis. Numerous computations show that generally the 
problem is solved faster if the simpler procedure is adopted, namely intro- 
duction into the basis of vector A, with the least evaluation A,. Here 6, is 
deterriined in each iteration only once and the number of computations in 
each iteration is considerably reduced. In the present section the compu- 
tational procedure is described with reference to this simpler way of 
choosing the vector to be introduced into the basis. 

The vector occupying the position of the basis on which 

6, = min = 

Xte >o *ik 
is obtained (& is the index of the vector to be introduced into the basis) is 
eliminated from the basis. If 6, is obtained for several vectors, any of 
these is eliminated; to be specific, we may decide to eliminate, e.g., the 
vector with the least position index. This simple rule does not eliminate 
the possibility of cycling. However, cycling in linear-programming prob- 
lems is an extremely rare phenomenon and the more cumbersome rule, 
which allows for the possibility of cycling (see 2-3), should be applied only 
once a cycle has been detected. Once the cycle has been eliminated, it is 
advisable to return to the simplified rule of determining the vector to be 
eliminated from the basis. 
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Having determined the index of the vector to be introduced into the basis 
and the position in the basis in which it is to be introduced (the position of 
the vector to be eliminated from the basis), we proceed with the computa- 
tion of the parameters of the current support program. Recurrence for- 
mulas (1.12) are applied to determine the basis variables cf the new sup- 
port program, the expansion coefficients of the restraint vectors expressed 
in terms of the new basis vectors, the evaluations of the restraint vectors 
with respect to the new basis, and the value of the linear form on the new 
program. These computations prepare the ground for the next iteration. 

2-2. We now describe the sequence of computations in a single iteration. 

Assume that the /-th iteration has been completed. The /-th tableau 
(see Table 5.1) is then filled, except for the last column. 


TABLE 5,1 


f-th tableau 


(¢ (t) (t) t 
i Cs, A, xy eat! X12 xt 
(t) (t) (t) t) 
2 C, A, *29 *31 +22 xi 
(t) (/) (¢) (i 
f Cy A, *r9 ¥F xre xy 
(ly (/) (t) (ft) 
m Cons A,,, *mo Xm ms xm 
m+i} — _ | L® | A®@) a@® AW) 


The tableaus are filled as follows. When there are no entries, a dash 
is written and if the entries are zeroes, the cells are left empty. 

In the first column (No.) of the tableau we give the row number, The 
first m rows are numbered So as to coincide with the basis positions. In 
the second column (C,) the coefficients c, of the linear form associated with 
the basis variables, are written. 

The basis vectors A, are entered into the column (B,). Inthe next column 
(A, = 8) we write the basis components x, of the support program (the coeffi- 
cients of the expansion of constraint vector B written in terms of the basis 
vectors). The columns A,, A,,..., 4, contain the coefficients x,, of the cor- 
responding expansion of the restraint vectors A, expressed in terms of the 
basis vectors. Column elements refer to the entries in the first m rows 
of the tableau only. In the last, (m+1)-th, row we write the evaluations AV) 
of the restraint vectors A, with respect to the basis. In the (m+1)-th cell 
of the column A,=B we write the value of the linear form L(x). We remind 
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the reader that in the previous section we used the notations 
A) = xine, h L(X®) =x,,,, e . 


The index / indicating the number of the iteration will be omitted wherever 
no confusion can arise. 

The columns A,, A,,..., A, (all the m+1 entries) constitute the princi- 
pal part of the tableau. The principal part of the tableau is, thus, a 
matrix I xt? |, where i=1,2,..., +1; /=0, 1, 2,..., a. 

Computations made in the (/+1)-th iteration are written in the column 6 
of the -th tableau and form the principal part of the (/+-1)-th tableau. 

The first stage of the (/+1}-th iteration starts with examination of the 
(m+ 1)-th row of the /-th tableau. To establish whether case (a) applies 
(optimal program), we must isolate the vectors with negative evaluations. 
If allA,=~x,,,,,20(/=1, 2,..., 2), the support program produced by the /-th 
iteration is an optimal program of the problem. Once this fact has been 
established, the solution process is terminated. Now let us assume that 
there are negative evaluations in the last row of the /-th tableau. To 
establish whether case (b) applies (unsolvable problem), we examine the 
columns A, with A,<0 and check the signs of the entries in these columns. 
If there exists at least one column A, with 4,<0 for all x,,q0 (i= 1, 2, ..., m), 
the problem is unsolvable (case(b)). Once this fact has been established, 
again the process terminates. 

Case (c) applies if each column A, with A,<0 contains at least one posi- 
tive x,, If so, we proceed with the second stage of the iteration. 

The vector A, with the last evaluation 


A,= i A, 


is introduced into the basis. After A, has been chosen, the last column of 
the i-th tableau, 6, is filled. In column 4 we write the ratio of the basis 
variables x,,(the entries in the first column A,=8) to the corresponding 
components x, (the entries in the column A,). Only those cells of the col- 
umn § are filled which correspond to positive x,. The least element of 
the column @ is denoted by 6, The basis vector As on which 6, is obtained 
should be eliminated from the basis. If the support program X**” is de- 
generate and, consequently, 6, is obtained on several basis positions, any 
basis vector in these positions is eliminated. If cycling occurs, apply the 
rule given in 2-3. 

The column A,, corresponding to the vector introduced into the basis, 
and the r-th column, corresponding to the vector eliminated from the basis, 
are marked, e.g., by aframe. The column A, and the r-th row are called 
the direction column andthe direction row of the transforma- 
tion, respectively. The element x, in the cell which is the intersection 
of the direction column and the direction row is calledthe direction 
element.* 

Having found the direction element, we proceed to fill the (#+1)-th 
tableau. Inthe rth cell of column By we write the vector A, which is now, 
according to its new position, denoted by A,, in the (/+1)-th tableau. In the 
other cells of the column By we write the same vectors as in the /-th 
tableau. The rows corresponding to the vector introduced into the basis in 
the preceding iteration and to the vector to be eliminated from the basis (the di- 
rection rows of the preceding and the present iterations)are markedby arrows. 


* (Also called pivot, pivotal, determining, or key element. ] 
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The principal part of the (/-+1)-th tableau is filled in according to the re- 
currence formulas (1.12). First the r-th row of the (/+-1)-th tableau is 
filled. According to the recurrence formula for the r-th row of the (f+1)}th 
tableau, the r-th row of the /-th tableau should be divided by the direction 
element x,,: 

x) x) 
Xj ase TE j/=9, Le Ze days (2.1) 
X rR 
To obtain the entries of the i-th row of the (/+ 1)-th tableau (/=1, 2,..., m, 
m+; ixr), we subtract from the i-th row of the /-th tableau the s-th row of 
the (/+1)-th tableau multiplied by <x?: 


x) 
(i+1) Ud) x xh) (i x 
Xp XE mu Xie = xi} — x (2.2) 
x Pk 


i=1, 2,..., m, m+1; i#r; j=0, 1, 2,,.., 2. 


Having written all the parameters in the principal part of the (/+1)-th 
tableau, the iteration terminates. Proceeding from the principal part of 
the (/+1)-th tableau, we carry out the (/+4-2)-th iteration according to the 
same rules used in the (/+1)-th iteration based on the /-th tableau. Con- 
siderations of §2 of the preceding chapter ensure that the iterative process 
is terminated either when an optimal program is reached or when unsolva- 
bility of the problem is established. 

All the tableaus, except one, of the algorithm are filled according to 
the same rules. The exception is the zeroeth tableau with which the com- 
putations start. The zeroeth tableau contains an additional row C in which 
the coefficients c, of the linear form are written. The initial basis and 
the basis components of the initial support program are assumed to be 
given. They are written in the corresponding positions of the columns By, 
andA,=8. Theentries x;,(i=1,2,...,m; j=1, 2,..., mare also assumed to 
be known. If the initial basis consists of unit vectors, x,,=a;,, x,,=06; In 
more complicated cases, various artificial methods for computing x;, are 
used. In some instances the x;, must be computed from systems of linear 
equations. 

If the initial program is not given in advance, then, applying the tech- 
niques of 58, we may, simultaneously, determine the initial program and 
all the relevant parameters to be written in the zeroeth tableau. 

The entries of the (m+ 1)-th row of the zeroeth tableau are computed 
from 


Lmsiy f= Qi*iso— Cp J=G, 1 2icasen (2.3) 


(We recall that c,=0and 
Xm+ry 0 = 2 X igls; =L (X).) 


Thus, to obtain the (m+1)-th row of the zeroeth tableau we subtract row 
C (the first row of the tableau) from the row product of columns A, and 
column C, (the first column on the left of the tableau). 

The principal part of the zeroeth tableau contains all the initial data re- 
quired for carrying out the first iteration, Formulas (2.3) can, obviously, 
be used for direct computation of the entries in the last row of any tableau 
of the first algorithm. 
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Since x,,,,, can be computed by two independent methods we have a 
means of control in each iteration. After a certain number of iterations it 
is always advisable to compare the entries in the (m+1)-th row of the 
tableau as obtained from the recurrence formulas with the values of the 
linear form and the evaluations A,as computed directly from formulas (2.3)3 
If the values of the linear form obtained by the two methods do not coincide, 
we first must check whether the entries of the column A, constitute a fea- 
sible program of the problem. If x,, do not satisfy the problem restraints, 
it is advisable to repeat all the computations starting with the iteration for 
which the previous check was made. Assume that the values of the linear 
form obtained by the two methods coincide, but the evaluations A, are dif- 
ferent for some /. In this case, the corresponding restraint vectors A, 
should be expanded anew in terms of the basis vectors. Several systems 
of linear equations differing only in the right-hand sides are conveniently 
solved by the Gauss method (see Appendix, 2-7 and /110/). 


Pe oe es | mevessemnrwescensoseceseneensernenens 7 
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FIGURE 9.1 


In Figure 5.1 we give a block diagram of a single iteration for the solu- 
tion of a linear-programming problem by the first simplex algorithm. 

2-3. To complete the discussion on the first algorithm, we now consider 
the rule applied to guarantee against cycling arising in degenerate linear - 
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programming problems. In computations according to the first algorithm, 
the basis vectors A, of the support program (it is assumed that the enu- 
meration of the restraint vectors is preserved in computations) are, ob- 
viously, chosen for the vectorsR,, R,,...,R,(see § 6 of the preceding chapter). 
The basis vectors satisfy the two conditions imposed on the system (R,, ..., 

.., R,). First, the vectors A, are linearly independent, second, the basis 
of the support program X is also the basis of some program of the e-prob- 
lem where 


Bie) = B +d a! Ay. 


The components of the support program of the e-problem, i.e., the coeffi- 
cients of B(e) expanded in terms of the basis vectors of program xX, are 
equal to x, (8) =x), el. 
Hence, the rule proposed in Chapter 5, § 6, for preventing cycling may be 
applied with R;=4A,. The basis vectors of the support program X are cho- 
sen for the system {R;} because during the process of solution of the prob- 
lem according to the first algorithm these vectors will always have to be 
expanded in the current basis. 

To apply the rule for avoiding cycling, columns 06’, 6’, ...are added on 
the right to the iteration tableaus corresponding to degenerate programs. 
In the degenerate case 

6,=— min —* 
Xu >0 a. 


is obtained simultaneously on several positions. We take R,=A, and write 


in column 9’ the ratio a for the i-values on which 6, is obtained. We 


choose 96,, the least of these numbers. The vector on which 6, is obtained 
should be eliminated from the basis. If there are several such vectors, 
the aforementioned procedure is repeated. In column 6” we write the 


x eo 
ratio 3. for the i-values on which 4 is obtained. Computations are con- 


tinued until the position of the vector to be eliminated from the basis is 
uniquely specified. 

2-4, We now estimate the bulkiness of computations according to the 
first simplex algorithm. The computer time required to solve the problem 
is determined, mainly, by the amount of multiplications and divisions to 
be carried out in the process of optimal-program determination. The 
operations of comparison, addition, and subtraction in universal computers 
are much faster than multiplication and division. The memory-address 
time is neglected here. Using the first algorithm, division is necessary 
twice in each iteration: Bis filling the tableau row corresponding to the 


new basis vector (x= = = Fi eee ) and when the column @ is computed 


(0 = ie with ty,>0). ee rise of the entries of the x,,; -th row requires 


aa divisions, and computation of the elements of the column 6 terminate 
after at most m divisions. The total number of divisions in each iteration 
of the first algorithm is, thus, at most n+1. Multiplication is necessary 

in the first algorithm only when transforming the tableaus according to the 
recurrence formulas (2.2): 


xSP me xf) — MO, 1, 2, 0.., mm mtd; 
ir, j=0, 1, 2,..., 2 
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The coefficients x{/*” corresponding to the basis vectors are not computed 


here (they are either zero or one). In each iteration there are, therefore, 
(n—m-+1)m multiplications. 

The number of iterations necessary to Solve the linear-programming 
problem varies depending on the problem under consideration and the initial 
Support program. There are no standard theoretical procedures for esti- 
mating the number of iterations involved in solving a general linear-pro- 
gramming problem. Experience shows that, as a rule, the number of 
iterations in various linear-programming problems ranges from mto 2m 

We repeat that for control purposes it is advisable, when computing ac- 
cording to the first algorithm, to compute the evaluations A, after a certain 
number of iterations not only from the recurrence formulas, but also 
directly from (2.3). This requires m(n+1) multiplications. If the evalua- 
tions of the basis vectors are not checked (A,=0 for i€/,), the number of 
additional multiplications necessary for control purposes is m(a—m+1), 
which is equal to the number of multiplications in a single iteration. The 
control procedure-under the first algorithm is, thus, fairly bulky. We 
know of no estimates for the optimum frequency for carrying out the compu- 
tational check. Detailed study of this problem, at least for particular 
classes of problems, is obviously of considerable practical interest. 


§ 3, Examples 


We shall illustrate the sequence of computations according to the first algorithm by two examples. 
Example 1, Maximize the linear form 


L (X) =3x,—x, + 8x, + 2x,—x, + 9x, 
subject to the conditions 

— 6x, + 9x, +3x, —2,— x,< 12, 

—4x,+3x,—3x,+ x,— x25, 
2x, + 8x, — 5x, + 6x, — 8x, + 4x, < 20, 
— x, —3x,— 4x, — 8x, +4x,< 10, 
5x, + xX, +2x,+ 4x, + 9x, +5x, <= 24, 

x,20, j=1, 2, anes 6. 


Toreduce the problem to canonical form, we introduce five additional nonnegative variables x, ..., %. 
Unit restraint vectors correspond to these new variables. The unit vectors A,, ..., A,, should, obviously, 
be taken as the initial basis. The entire solution process is given in Tables 5.2, The tables consist of 
tableaus corresponding to the separate iterations of the simplex procedure, and they are numbered according 
to the sequence of iterations, 

The principal part of the zeroeth tableau (except for its first and last rows) is filled with the correspond- 
ing constraint vector (A,) and restraint vectors (A,, Ay, ..., A,,). The components of the constraint vector 
are written in the tableau as the basis components of the initial support program, and the components of the 
restraint vectors as the coefficients of the vectors A, expressed in terms of the unit basis vectors, 

The column Cy remains empty, since zero coefficients correspond to the additional variables in the 
linear form. 

The last row of the zeroeth tableau contains the evaluations A, of the restraint vectors. The parameters 
A;=%Xm+i,7 are computed from (2,3), In our case, Cs, =0. Therefore, in the Ath row, we write the ele- 
ments of the first row (C) with reversed sign. The linear form vanishes on the initial program. 

Among the evaluations A, of the restraint vectors there are negative numbers, The initial support pro- 
gram is, therefore, not optimal. In the expansion of each of the restraint vector with negative evaluations 
there are positive xj; Hence, there is no reason to assume that the problem is unsolvable, We thus have 
case (Cc). 

The least evaluations A,;=—9 corresponds to Ag, The vector A, should be introduced into the basis. 
The last three elements (Xj, Xeg, Xs.) Of the direction column A, are positive. The 6-values are therefore 
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computed only for the third, fourth, and fifth entries of the column @. The least entry in the last column 
corresponds to the fourth basis position, being equal to 


This indicates that the fourth row is the direction row of the transformation and that the vector A,, situated 
in this row is to be eliminated from the basis. 

We now proceed to compile the 1st tableau corresponding to the first iteration, In the column By the 
fourth entry is A,, which has replaced A,,. in the fourth position of the basis, The corresponding coefficient 
c, of the linear form is written in column Cy. The principal part of the 1st tableau is filled from the data 
of the preceding zeroeth tableau according to the recurrence formulas 


, x ve 

y= Hy He i=0, 1, 2,3,5,6; f=0,1,2,..., 1; (3.1) 
, x 

nya 7=0, 1, 2, ..., HL (3.2) 


The direction element is x,,=4. Hence, the fourth row of the lst tableau is obtained from the fourth row 
of the zeroeth tableau when the latter is divided by 4. The remaining entries in the principal part of the 
1st tableau, the components of the support program and the evaluations of the restraint vector included, are 
computed from recurrence formulas (3.1). For instance, 


—4(—))_ 
Sage 


’ x, 

he ane ed ea 
‘ ‘ x, 10 (— 1 

4, 4,,.= yy — 28 y= — 75, 


—3(—9)_ 


r — 5.75. 


, ’ Xas 
A,=% = %a— Xe = l— 
Ge 


The iteration terminates when the principal part of the tableau is filled. The next iteration is carried out 
following the same rules, In each tableau arrows indicate the vectors introduced into the basis in the pre- 
ceding iteration and the vectors to be eliminated from the basis, In the 7th tableau all the evaluations A, 
are nonnegative. This indicates that the program X") is a solution of the problem, 

To check the computations, it is advisable, at some stages, to determine the estimates A; not only from 
the recurrence formulas (2.2), but also directly from (2.3). Let us compute, e.g., A, in the lst tableau: 


a 
A,= D>) Xie Gs, — Cp = 8.25.0— 4.75.0 + 11.0—0.75.9 + 4.75.0—(— 1) = —5.75. 
é=1 


The same result was obtained from the recurrence formula (3. 1). 
Example 2, Tables 5.3 give the sequence of operations leading to the solution of a problem requiring 
minimization of the linear form 


£ (X) =— 9x, + 8x,—5x, + 3x,— 8x,— &x,, 
subject to the conditions 


6x, + 2x, + 2x, + 3x, <, 24, 
2X, +3x, +5x,+9x, <, 30, 
9x,++, +5x,+ %, < 40, 
6x, + 2x, + x, < 36, 

+ 4x, + 8x, <. 20, 

3x; +x, + 8x, + 4x, + 4x, < 48, 


xj=>0, fb 2, sag Oo 
This problem is equivalent to maximization of the linear form 
L (X) =9x, —8x, + 5x,— 3x, + 8x, + 8x, 
subject to the same conditions. 
Tables 5.3 do not call for special explanations, The solution is obtained in five iterations (the zeroeth 


iteration is not considered as an independent step; the principal part of the zeroeth tableau contains data 
preparatory for the first iteration). 
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TABLE 5.3 (0-5) 
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TABLE 5,3 (continued) 
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§ 4. The coordinate form of the simplex method 


4-1, In the preceding chapter, in our discussion of the simplex method, 
we started with such concepts as restraint vector, constraint vector, sup- 
port program basis. The transformation from one support program to the 
successive one was effected by replacing one of thebasis vectors by another, 
and the simplex parameters in each iteration were determined by expanding 
the restraint vectors and the constraint vector in terms of the basis vectors. 
The name ''vector form" suggests itself for describing this approach to the 
simplex method. 

There are also other approaches to the description of the method and 
construction of the corresponding algorithms. In particular, the structure 
of the tableaus of the first algorithm is directly determined by the so-called 
coordinate form of the simplex method. An analysis of the coordinate form 
cannot contribute anything essentially new to the fundamentals of the method. 
Nonetheless, a description of a new form of the method, with due emphasis 
on the individual details, produces fresh analogies and is thus conducive 
to the assimilation of both the theoretical principles of the method and the 
computational procedures. 

Consider the linear-programming problem in canonical form. 

Maximize the linear form 


L(X)= 2 ex, (4.1) 
subject to the conditions 
a 
Pe Ayty=B, (4.2) 
x,20, j=l, 2, ..., a. (4.3) 
Let X°=(x, ..., x*) be some support program of problem (4.1)-(4.3) with 
basis A,, ..., As; let the set of indices s,, ..., s, be denoted by /,, and the 


matrix of the basis vectors A,(j€/,) by Ay. Thus 
Ay =(A,, @evy A,,). 


Let Xy=(x2,, +... *),,)7 be the vector whose components are the coefficients 
in the expansion of the vectors A,(j=0, 1, ..., 2) in terms of the basis of 
program X°: 
or, equivalently, 

My = AXA). 
Multiplying restraints (4.2) on the left by Ax}, we obtain 

n 

oa Xi) =X, (4.4) 


The vectors x, corresponding to the basis components of the support pro- 
gram X are unit vectors. Therefore, (4.4) can be rewritten as follows: 


fat By = hey ba 1 Oya 
El 


As usual, we establish the relation between the notations of the basis 
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variables of the support programs and the positions of the corresponding 
vectors in the basis. We have 


X y= XY x? x), i=1, 2, ..., m. 
=e Bay (4.5) 
Applying the above relationships, we rewrite (4.1) as follows: 


Rn m 
L(X)= Que) = 26 sek ag + PT? = 


whence 
L(X=L (X*) — pa AS x). (4.6) 
él, 
Let 
L (X) = Xsqs3 L(X) = Xn, 0° } 
A® = x 


{ mi, f' 


(4.7) 


In these new notations, relationships (4.5) and (4.6) are combined into a 
Single system 


Fume Bey Fat Boom mtd. (4.8) 


Formulas (4.8) give an alternative representation of the linear form (4.1) 
and restraints (4.2) of a problem where the restraint matrix and the coeffi- 
cients of the linear form are replaced by the parameters of a given support 
program. 

We now compare (4.8) with the zeroeth tableau of the first algorithm. 
We see that the parameters characterizing the initial support program are 
written in the (#+1) rows of the tableau in the same order as they appear 
in equalities (4.8). The last (m+ 1)-th relationship of system (4.8) (in pre- 
vious notations, formula (4.6)) shows that if all AJ>0, then for x,>0 (//,) 
the linear form L(X)may not exceed L(x); the support program X° proves 
to be the optimal program of the problem (case (a)). If for at least one 
Av<O all ~ <0 (i=1, 2, ..., m), we see from (4.8) that when z, increases to 
infinity programs xX, on which L(X) increases to infinity, are obtained. Thus 


the condition 
Xmn+1, p= As <0 


xt, <0 


indicate unsolvability of the problem (case (b)). 

Now we assume that for all the indices / for which Aj<0 there exists at 
least one x? >0 (case(c)). We shall show how to transfer to program X’ at 
the same time increasing the value of the linear form. 

Take k&€/, such that 


and 


Xnt+ik= ARP<o. 


Now, express one of the x, in (4.8) in terms of the other variables. Let 
the number, i, of the equation from which x, is taken be r. Solving this 
equation for x, we obtain 


X= X, pa Xp php 
i€l, 
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where 


x’ _ 70 , x} 
=o? kf yo? 
Xrh add 


and the set of indices /, is obtained from /, when k is substitutedfor s,. 
Let /,={s',s\,...,s,} where s;=s,for is4r, and s’=k, Moreover, let 


Then 


xy *n— 2 xp Xp (4.9) 


These new notations are justified since the variables x, and x, are obtained 
from the r-th equation of system (4.8). 

Substituting the expression for x,=x,’ into the other equations of system 
(4.8), we obtain , 


Xe =z Xo x! Xx, i=1, 2, ..., m, m+; 
ae A. cd 
ixkr, (4.10) 


where 


Combining (4.9) and (4.10), we rewrite the transformed system (4.8) in 
the form 


xy =x — YP xx, 
= Ho Dy Mus (4.11) 
where 


tf] 
“tye, he for ir, 
6 

il for i=r, (4.12) 
Xr 


i=l, 2, Bees m+; J=0, 1, 2; ceoyr A 


’ 


Until now r was an arbitrary index. However, to ensure nonnegativity of 
the components x), (/=1,...,m), r must be chosen subject to the condition: 


x20, t=1, 2, ..., m. 


It follows from (4.12) that, for x<0, x/, are always positive. For ,>0, 
x, are positive if 


0 
*r9 


X19 


eth 
Hence r must be chosen subject to the condition: 


*, x; 
—= min —S (4.13) 


We now write (4.12) for i=m+1 and /=0 in the previous notations: 


8 
L (X’) = L(x) ——2 Ae, 
rk 


where X* and X’ are support programs of problem (4.1)—(4.3) with the basis 
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eomponents xf and xy (é=1,2,...,m) respectively. For given & and rand 
2, =6,>0 

: L(X) DL (X*). 
To sum up: having suitably chosen the indices & and r, we applied the 
identity transformation to (4.8) thus obtaining (4.11) and then passed from 
program X*>0 to program X >0, in which L(X’)>L(X°), 

The structure of system (4.11), like that of system (4.8), is such that if 
the variables x, over which summation is carried out (extrabasis variables) 
are taken equal to zero, we obtain a Support program and the value of the 
linear form of problem (4.1)-(4.3). 

Let us compare system (4.11) with the /-th tableau (Table 5.1) for /=1 
of the computational procedure according to the first algorithm. We See, 
in the tableau, that the support-program parameters are written in the 
same order as in the right-hand sides of equations (4.11). The first-algo- 
rithm tableaus, in fact, reflect the successive transformations associated 
with the solution of a system of linear equations by the complete elimination 
method (the Gauss method). The only feature which converts the Gauss 
method of solution of a system of linear equations into a method of solution 
of the corresponding linear-programming problem lies in the special choice 
of the direction element x,,(the choice of the indices r and ). 

We assert that the simplex method actually specifies a process for sol- 
ving a system of linear equations — the problem restraints —by the complete 
elimination method supplemented by a special rule for choosing the direc- 
tion element. The latter ensures succession of programs (the selection 
rule for the row r) and monotonic increase of the linear form (the selection 
rule for the column k). As we shall see below, other finite methods of 
linear programming are also reducible to the complete elimination method, 
though modified by other selection rules for the direction element. 


§ 5. The second simplex algorithm 


5-1. From the second form of the optimality test we obtain a variation 
of the simplex procedure, the so-called second algorithm or the 
inverse-matrix method. 

A variation of the simplex method, close to that described here, was 
first applied by L.V. Kantorovich to one of the particular problems of 
linear programming /66, 63/. Later (1951), this algorithm, as one of the 
realizations of the so-called method of decision multipliers, was applied 
to the solution of the general linear-programming problem /67/. 

We shall consider the linear-programming problem in canonical form, 
problem (4.1)-(4.3). Let X be a support program of the problem with the 
basis A,,, As, ..., As, Let /y indicate the set of indices of the basis vectors 
(the numbers of basis variables). 

We construct the square matrix Ay of order m. The columns of Ay, are 
the basis restraint vectors. The determinant of A,, therefore, doesnot 
vanish, and the inverse Ax' exists. We shall show that, using A x' and 
the problem restraints, we obtain a fairly compact procedure for computing 
the parameters required in subsequent program improvements. 
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To use the simplex method, we must know, given an initial support 
program and its basis, how to choose in each iteration a restraint vector 
A, to be introduced into the basis and a vector A, to be eliminated from the 
basis. The vector A, is a restraint vector with a negative relative evalua- 
tion A,(generally, A,=minA,). The vector A, is the vector on which the 


ratio oat for x;,>0, obtains its least value. Therefore, to pass from a 


given program to another program ensuring a higher value of the linear 
form we must know how to compute in each iteration the basis components 
x, Of the support programs, the evaluations A, of the restraint vectors 
with respect to the support-program basis, and the coefficients x, in the 
expansion of the vector A,, to be introduced into the basis, in terms of the 
vectors of the current basis. 

According to the first algorithm, to determine these parameters in 
each iteration we had to compute the coefficients x,, of all the restraint 
vectors A, written in terms of the basis vectors. As we shall see below, 
all the parameters necessary for program evaluation and for transferring 
to another, better program can be obtained by step-by-step transformation 
of the elements of the inverse matrix Ax’. 

Indeed, the basis components of the support program are the elements 
of the product of matrix Ax'and constraint vector 8. The evaluations A, 
of the restraint vector can be computed (see Chapter 4, § 2) from 


Ay=2,—¢,= 2 aihj—e)=(A, A) — ep 
jJ=1, 2, ..., a, (5.1) 


where the parameters 4, for /€/y satisfy the equations 


py. aj Ap=Cp 


or, equivalently, iat 
AAy = Cy, (5.2) 


where Cy is a row vector comprising the linear-form coefficients which 
correspond to the basis variables. Hence, 


A=C,Ax', (5.3) 


With the aid of (5.1) and (5.3) we can compute the evaluations A, of the re- 
straint vector from the elements of the inverse Ax’ and the problem re- 
straints. 

The coefficients x, in the expansion of the vector A, in terms of the 
current basis are the elements of the product of matrix Ax'and restraint 
vector A,. The coefficients x, together with the basis components of the 
support program specify the vector to be eliminated from the basis. More- 
over, the signs of the coefficients x, give an indication as to whether the 
problem is solvable. Obviously, with the second algorithm unsolvability 
of the problem, if it is indeed unsolvable, is usually detected later than 
with the first algorithm, where the signs of x,,can be established for all 
the vectors A, with negative evaluations. 

Thus, to use the simplex method, it suffices to know how to compute in 
each iteration the inverse Ax’ of the matrix constructed of the current 
basis vectors. 

The column entries of the inverse Ax' are, conveniently, taken to be 
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the coefficients es; in the expansion of the unit vectors e; G=1, 2, ..., m)in 
terms of the basis vectors. In §1 we derived recurrence formulas (1.21) 
relating the parameters e,, of two successive iterations. We recall that 

e,, for i=1, 2, ..., m, j=1, 2, ..., mare the coefficients in the expansion of 
the unit vectors in terms of basis vectors; e,=.x, are the basis variables 
of the support program; em4,,;=A, are the evaluations of problem restraints 
with respect to the given program; em4:,,.=A4,=L(X) is the value of the linear 
form. 

Recurrence formulas (1.21) constitute the basis of the second simplex 
algorithm. The role played by the inverse matrix Ax' in the computational 
procedure justifies the name of the inverse-matrix method some- 
times given to the second algorithm. 

9-2. Below we give the sequence of computations under the second al- 
gorithm. Solution of the linear- programming problem according to the 
second simplex algorithm involves successively filling in a system of prin- 
cipal tableaus and an auxiliary tableau. 

The principal tableaus have the following structure (see Table 5.4). 


TABLE 5.4 


The principal &-th tableau 


(f) (h) (é) (!) () 
A, 10 en ers re Cim ll *sk 
(4) () (2) (!) (f) 
er0 Cs Cos see em Xek 
1 l rf (2) (1) 
a ra ae ane erm xrk 4 
i rf i (7) (4) 
ay ef) ey -++ | 2mm || *me 
i rs) t) (4) (t) 
— fu ec ae 


In the first column (No.) the number of the row is given. In the second 
column (C,) the linear-form coefficients ¢,,, corresponding to the basis 
variables, are written. Under By we give the basis vectors A, The next 
(m+-1) columns constitute the principal part of the tableau which contains 
the coefficients ej) of the expansion of vectors e, (j=0, 1, ..., m) in terms of 
the basis vectors. Here e,=8 is the constraint vector and e, is an m- 
dimensional unit vector with unity in the /-th position. The (m+ 1)-th entry 
in each column in the principal part of the tableau is the value of e() |: 


/ 
an J EX) for J=0, 
PY a for s=1, 2, ..., m. 
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On the right of the principal part of the tableau we have the column A, in which 
the coefficients «x of the expansion of the vector introduced into the basis in 
terms of the basis vectors is written. The (m+1)-th entryinthe column A, is 
the evaluation Af of the vector A, with respect to the basis. The entries 
in the last column of the principal tableau, column §, are used for choosing 
the vector to be eliminated from the basis. The column 6 is filled, as in 
the first algorithm, with the ratios of the basis components e( of the pro- 
gram to the corresponding entries x? in column A,. The entries in column 
§ corresponding to «x{) <0 are crossed out. 

The upper part of the auxiliary tableau (Table 5.5) contains the augmented 
restraint vectors (the restraint matrix and the linear-form coefficients) 
and the constraint vector B. 


TABLE 95.5 
The auxiliary tableau 


In the lower part of the auxiliary tableau evaluations of the restraint 
vector with respect to each of the support-program bases obtained in the 
process of solution are written in the rows. The rows in the lower part 
of the auxiliary tableau are numbered according to the number of iterations 
(more precisely, by the number of the principal tableaus). In row 0 we 
write evaluations of the restraint vector A, with respect to the basis of 
the initial support program. With each iteration a new row AY) is added to 
the auxiliary tableau. The solution process is terminated when a row of 
nonnegative elements appear in the auxiliary tableau, or when a column 4, 
with nonpositive x? is obtained in a principal tableau. 

Let us now consider the sequence of computations in one iteration. 
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Let the [-th iteration be completed. The /-th principal tableau (with the 
exception of the last two columns A, and 4) andthe A row of the auxiliary 
tableau are filledin. In the first stage of the ({+1)-iteration, the A -th row 
of the auxiliary tableau is examined. If all As’ >0, the support program ob- 
tained in the /-th iteration is optimal (case (a)). We now assume that there 
are vectors with negative evaluations. The vector A, with the least evalua- 
tion A} is introduced into the basis, and the coefficients x{? in the expansion 
of A, in terms of the basis vectors are written in column A, of the /-th 
principal tableau. Each x‘) is obtained by multiplying column A, in the 
upper part of the auxiliary tableau (positions1,2,...,m)by the i-th row of the 
t-th principal tableau (j=1, 2, ..., m): 


= De (5.4) 
j=1 
The (m+1)-th entry in column A, is the evaluation Af? of this vector. 

Examine the column A,. If all x) <0 (i=1, 2,..., m) the problem is un- 
solvable (case (b)). 

We continue with the second stage of the iteration if there exists at least 
one positive coefficient x<). (Observe that under the first algorithm the 
vector A, to be introduced into the basis was chosen in the second stage of 
the iteration.) 

In the second stage we determine the vector to be eliminated from the 
basis, compute the new support program andthe initial data for the next 
iteration. In column 6, as inthe first algorithm, the ratios of the basis 
components of the support program (elements of column e,) to the corres- 
ponding components x of column A, are written. The entries in column 6 
corresponding to «<!)<0 are crossed out. The least element of column 9 
is denoted by @,. The basis vector A, on which §, is obtained should be 
eliminated from the basis. If program xX is degenerate and 6, is obtained 
on several vectors, any of these can be eliminated from the basis, e.g., 
the vector with the least index. If cycling occurs, the rule given in 5-5 
should be applied. 

The row r of the /-th principal tableau is singled out. The element x} 
at the intersection of the r-th row (the direction row) and the column 
A, (the direction column) ofthe principal tableau is called the direc - 
tion element of the transformation of the /-th principal tableau into 
the (/4-1)-th tableau. 

In the r-th cell of column By of the (J+ 1)-th principal tableau we write 
the vector A, According to its position in the basis, the vector 4A;, is de- 
noted as A, in the (/+1)-th tableau. In the rest of the positions of the 
column B, the same vectors as in the /-th tableau are written. 

The principal part of the (/+1)-th tableau is filled by using the data of 
the /-th tableau and recurrence formulas (1.21). To obtain the r-th row in 
the principal part of the (/+1)-th tableau, we divide the r-th row of the /-th 
tableau by the direction clement x: 


(f 


é 
a = j=0, 1, 2, o00) m. (5.5) 


f 


To fill the i-th row of the principal part of the (/+1)-th tableau (i=1, 2, ..., 
ooe, m+il; i#r), we subtract, according to the recurrence formulas, the 
r-th row of the (/+1)-th tableau multiplied by +() from the i-th row of the 
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i-th tableau: (t) 


é 
i TL vat) — ptt) _ pl l 
elit? el) — th att = el) — elt ap, 
rk 
i=1, 2, ..., m, m+1; j=0, 1, 2, ..., m. (5.6) 


To complete the (f+ 1)-th iteration, the evaluations of the restraint vec- 
tor with respect to the basis of the new support program, must be written 
in the A“*”-th row in the auxiliary tableau, 

The A“+-th row is filled according to formula (5.1), The linear-form co- 
efficients c, are written in the (m+ 1)-th row of the auxiliary tableau. The 
components a,, of the vector A, occupy the first m positions of the column 
A, in the auxiliary tableau, and the components A, (=1,2, ..., m) of the vector 
A fill the last row in the ({4+1}th principal tableau. Thus, to obtain the 
A“+)_th row we must subtract the row (C) of the auxiliary tableau from the 
product of matrix A (the upper part of the auxiliary tableau) and the last 
row of the ({+1)-th principal tableau. 


SOOO Ee ROe (Fics OOHOHe POS OCEEHTOOSEODS- Se TOE SESTESSSTOSOSAOTSOTSS STOVE SEHES TEESE DEDEOHEOSSOEES 


_no: Case (a) 


s 


Test A;>0 
i (optimal program) 


yes| 


Choose vector A, : 
and calculate jz 


Ist stage 


7 
Test xj, > 0 no i_, Case (b) 
j(unsolvable problem) 


Establish the positions on 
which 46, is obtained 


Choose the direction row 


2nd stage 


Transform the principal 
part of the tableau by re- 
currence formulas 


FIGURE 5,2 
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The A“*"-th row of the auxiliary tableau and the principal part of the 
({+1)-th tableau contain all the initial data for the(/+2)th iteration. Subse- 
quent iterations follow the same outline. As shown in the discussion on the 
theoretical principles of the method, a finite number of iterations will 
produce an optimal program or establish unsolvability of the problem. 

All the tableaus according to this algorithm are filled following the same 
rules. Some exceptions arise in compiling the zeroeth principal tableau. 
In column e, we write the basis components of the initial support program. 
The first m entries in columns e,,e,, ..., e, are obtained by direct inversion 
of the basis-vector matrix of the initial support program, The last entries in 
columns e,, ..., e, arecomputed from formula (5.3), (We recall that the col- 
umns of matrix Ax' arethe columns e, of the zeroeth tableau.) Thus, the last 
row of the principal part of the zeroeth tableau is filled with the products of 
column C, and columns e, (j=0,1,...,m) of this tableau. 

Figure 5.2 shows a block diagram of a single iteration of the second 
simplex algorithm. 

In order to check the computations carried out under the second algo- 
rithm, we should, obviously, use the two methods for determining A, Inany 
iteration, the parameters 4, can be computed directly (as in the zeroeth 
tableau) or from recurrence formulas (1.21). All that has been said on the 
technique of control in connection with the first algorithm applies in this 
case too. 

9-3. We now estimate the bulkiness of computations in one iteration of 
the second simplex algorithm. We have already observed that this aspect 
of the computational procedure should be evaluated in terms of the number 
of divisions and multiplications necessary in each iteration. 

Under the second algorithm, division is necessary twice in each itera- 
tion — when computing the row in the new principal tableau corresponding 
to the vector newly introduced into the bases (see (5.5)) and when computing 
the column 96. In the first case there are m-+41 divisions, and in the second 
there are at most m. Multiplication under the second algorithm is neces- 
sary three times: first, when transforming the principal tableau according 
to formulas (5.6), second when computing the evaluations A, from (5.1), 
and third when determining the coefficients x,;, in the expansion of the vec- 
tor A, introduced into the basis in terms of the basis vectors. The trans- 
formation of the principal part of the tableau requires m(m+1) multiplica- 
tions (in formulas (5.6) i=1, 2, ..., mm+1; i#r; j=0, 1, ..., m). The compu- 
tation of A, for each restraint vector A; involves m multiplications and 
there is a total of n—m evaluations of the restraint vector to be computed 
(as the number of extrabasis variables; the evaluations of the basis 
variables need not be computed, since they are all zero). Hence, computa- 
tion of all the evaluations in one iteration involves m(n—m) multiplications. 
Finally, to compute all the x, (j=1,2, ..., m)from formula (5.4), another m' 
multiplications must be carried out. We thus see that each iteration under 
the second algorithm involves at most 2m-+1 divisions and 


m(m-+1)+m(n—m)+ m= m(n-+m-+1) 
multiplications. To check computations, it is advisable to compute the 
parameters A,, at fixed intervals, not only from the recurrence formulas, 
but also directly from formulas (5.3). Each control stage thus involves m' 


additional multiplications. 
Comparing the bulkiness of the individual stages in each iteration, it is 
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necessary to make the following comments concerning the choice of the 
direction element. 

For na>m, the computation of evaluations becomes the most lengthy 
operation under the second algorithm. All the evaluations must be comput- 
ed so that the least A,can be chosen. The vector A, for which 


is introduced into the basis. There will, obviously, be far less operations 
if the evaluations are computed until the first negative A, is obtained, and 
then the vector corresponding to this evaluation introduced into the basis. 
The number of iterations may increase; however, this is, generally, more 
than compensated for because the bulkiness of computations in each itera- 
tion is reduced. In each iteration the order in which the evaluations are 
computed is, generally speaking, arbitrary. We suggest the following se- 


quence. For the initial program, compute the evaluations A,, ..., A,,, where 
A,, is the first negative evaluation. For the next support program, com- 
pute Apa -.. A;,, Where A,, again is the first negative number, if all A, >0 


for g=p,+1, p,+2,..., 4, proceed with the computation of he ae 

5-4, The working memory is the bottleneck of modern computers. We 
shail outline a modification of the second algorithm, which enables the 
memory storage space to be used more economically /51/. 

The preceding computational procedure requires that in each iteration 
the entire inverse matrix Ax'=|/e,,|| be stored inthe memory. The so- 
called product form of the second algorithm (or product 
form for the inverse in the simplex method) involves 
storage of much less data. 

The product form is based on the following considerations, Let X and 
XxX’ be two successive support programs of the problem. Let the correspond- 
ing matrices of the basis vectors be A, and Ax: 


Ay =(Au,, eee, Ag, eee, A,,)s Ax = (As,, eney Ay, oeey A,,). 


It can easily be verified that the inverse matrices are related by the equa- 
tion 


Ax! = EAR, (5.7) 
where 10... 0 
Ol ... Vex 0 
EF’=|100 ... yy, -.. 0 fl’ 


x 
Yua=— i. f=1, 2, ..., m i#kh; Ina 


Relationship (5.7) is equivalent to the application of recurrence formulas 
(1.21) for /=1, 2, ..., m; j=l, 2, ..., m. 

Generally, the solution of the linear-programming problem starts with 
the unit basis. This corresponds to the unit matrix —. Therefore, after 
the first iteration, when the basis vector A, has been replaced by the vec- 
tor A,,, the inverse of the basis matrix of the new support program xX’ can 
be determined from 
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and, after the /-th iteration, from 
Avy = ECE ... E'B, (5.8) 


The matrix & is specified by the m+1 number (r, y,.,..+;Yas)) Hence, for 
i<m keeping the inverse matrix in the form (5.8) less of the memory is 
occupied than in the ordinary procedure, where mxm numbers e!? must be 
stored in each iteration. 

As already indicated, from experience we know that the number of itera- 
tions required for solving the linear-programming problem is, generally, 
of the order m. It should, therefore, be expected that the product form 
will almost always prove more compact thanthe original form of the second 
algorithm. In those cases when the number of iterations is greater than m 
and for some reason it is inexpedient to transform the inverse of the basis 
matrix of the support program X” to the unit matrix*, we should, starting 
with the (m+ 1)-iteration, revert to the original procedure of storing the 
inverse. 

Let us now outline the computational procedure for a single iteration in 
the product form of the second algorithm. 

Assume that the /-th iteration has been completed and it has been estab- 
lished that X” is not an optimal program but there is no reason to suspect 
that the problem is unsolvable. The vector to be introduced into the basis 
and that to be eliminated should, obviously, be determined by the general 
Simplex method. The components of the current support program are com- 
puted, as in the original form of the second algorithm, from recurrence 
formulas (1.21). 

The evaluation vector of problem restraints (A) is computed in the 
(i+ 1)-th iteration as follows 


Ay cm CEE". oe E" BE, 


The evaluation vector of the restraint vectors is determined as always under 


the second algorithm from 
A=AAy—C, 


Finally the coefficients x; in the expansion of the vector A, to be intro- 
duced into the basis in terms of the basis vectors are determined from 


AGH. BIB on BA 


Here 


(i4+1 f+1 (i+14)), (0) T 
ARTY (048), SED); Ag = AR =(G.gs 00s Ba) 


We see that the product form of the second algorithm not only economizes 
on memory storage space, but also in some cases essentially reduces, at 
least in the first iterations, the bulkiness of computations. 

5-5, Let us consider the specific features of the second algorithm in 
connection with the solution of degenerate problems. We have already in- 
dicated that unless cycling has been detected it is advisable to employ the 
simplified rule for determining which vector is to be eliminated from the 
basis. When acycle arises, the index r should be computed by the rule 
given in Chapter 4, §6. The unit vectors e, suggest themselves as the vec- 
tors R;, since in each iteration we compute the coefficients of e, when 
* Transformation of the inverse Ay? to the unit matrix in any single iteration corresponds to transition to the 

equivalent problem whose restraint vectors and constraint vector are obtained from the restraint vectors and 
the constraint vector of the initial problem when these are multiplied by Ay‘ on the left. 
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expanded in terms of the basis vectors. This system of vectors is linearly 
independent. The vectors ey however, need not satisfy the second require- 
ment to be filled by the system R, The coefficients in the expansion of 


the vector e,+ 5 ee, in terms of the basis vectors may prove to be negative. 
=1 


In the principal part of some tableaus a row whose first nonzero entry is a 
negative coefficient e,;,(0<i<m) may appear. To avoid this, it is advisable, 
once cycling has occurred, to introduce into the principal tableaus an addi- 
tional column W comprising only positive numbers, e.g., w;=1,i=1,2,...,m, 
and to take R,=A,W, R,=e,,..., Ry=e_,, Rig, —Cty,,---, Rya=elg Here Ay is the 
basis matrix; the index ¢ is chosen so that the ¢-th coefficient of the vector 
AyW is nonzero. The resulting system of vectors, obviously, satisfies all 
the conditions imposed on R;. It must be remembered that when passing 
from tableau to tableau the column W should also be transformed according 
to the general rules. 

The additional column becomes unnecessary if the rule for avoiding 
cycling is applied at the outset under the second algorithm. The vectors 
e, (j=1,...,m) may be used as the system R; ({=1, 2,...,m) also if, when 
cycling is detected, the inverse matrix is transformed to the unit matrix. 
It seems, however, that transformation of the additional column VW in 
several successive iterations will involve less computational work on the 
whole than application of the complex degeneracy rule throughout the solution 
process. Transformation to the unit basis is also rather tedious. 

The position of the vector to be eliminated from the basis is determined 


as follows. Let §,=min oe be eas on several positions. We compute 
é 


the minimum of the ratios 6;= ra for the indices ion which 6, is obtained 

s 
(or min , if column W is not required). If here 6, is not unique, we com- 
pute he ratios 6,;= on (or =o, if column W is not introduced), etc., until 


the minimum of the ratio is obtained on a single vector. This vector is 
then eliminated from the basis. 


§ 6. Examples and comparative evaluation of the algorithms 


6-1. We shall now illustrate the computational procedure of the second simplex algorithm by the two 
examples discussed in § 3 in connection with the first algorithm, 

In the principal tableaus, Tables 5,6, and in the auxiliary tableau, Table 5,7, we give the process of 
solution of Example 1 (see § 3). Tables 5.6 are divided into 8 tableaus, The principal part of the zeroeth 
tableau contains the initial data for the first iteration, The last two columns and the last row of the principal 
and the auxiliary tableaus, together with the principal part of the subsequent principal tableau and the A-th 
row of the auxiliary tableau correspond to the current iteration of the algorithm, In the upper part of the 
auxiliary tableau the components of the constraint vector and of all the restraint vectors are written. In the 
(m-++1)-th (sixth) row the corresponding coefficients of the linear form are written, The A rows in each 
iteration are filled from formulas (5.1). The initial basis comprises the unit vectors corresponding to the 


additional variables x,, X,,...,%,;. The corresponding linear-form coefficients are all zero, Therefore, in 
the principal zeroeth tableau all 4j=0 and in the row 0 of the auxiliary tableau A,=—cy The vector A, 
with the least evaulation (A,=—9) is to be introduced into the basis, 


Inthecolumn Ay, ofthe principal zeroeth tableau the coefficients x;, in the expansion of A, in terms of 
the basis vectors are written. Since the initial basis consists of unit vectors, x;,=a;,. In the (m+1)-th cell 
of the column Ag we write the evaluation A, of A, Column A, contains positive components, We thus 
have case (c). 
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We now proceed to determine the vector to be eliminated from the basis, In column @ we fill the cells 
for which xj>0 (cells 3, 4, and 5). The least value ¢=0,==2.5 corresponds to the basis vector Aj, 
occupying the fourth position in the basis. The fourth row is, thus, the direction row of the transformation, 
and the vector A,, is to be replaced by A,. The direction element of the transformation is x,==4, occu- 
pying the intersection of the fourth row and column Ag. 

The principal part of the 1-th tableau is computed from the elements of the principal part of the zeroeth 
tableau with the aid of the recurrence formulas. The entries of the fourth row in the I-th tableau are equal to 
the corresponding entries in the fourth row of the zeroeth tableau divided by the direction element x,=4. 
All the remaining elements of the principal part of the tableau (including the components of the column @ 
and the row 4) are computed from formula (5,6), For example, 


’ 10-4 
X aq = Cyg = Cy X= 20 ——7- = 10, 
i 5 
04 ly — Ht xy 0— Pa I 25, 
ey ’ ? ° @, 10(—9 
L (X’) = Cm. 3,9 =A = Cen = Cea — = Kes = 0 — is ) 205. 
ae 


Here x,,=A, is the evaluation of the vector A, introduced into the basis. 
Now, using (5.1), we compute the first row (A’) of the auxiliary tableau, Table 5.7. We have, for 
instance, 
A, =(0-0—3-046-0—8-2.25 +.4-0)—2—— 20. 


A : is the least element in the A’ row, A, is thus introduced into the current basis, Subsequent iterations 


follow the same rules. For control purposes it is advisable in some iterations to compute the restraint evalua- 
tions A; not only from recurrence formulas (5.5), but also directly from (5. 3), 

In Tables 5.8 and 5.9 we present the sequence of computations leading to the determination of the 
optimal program of Example 2 (see § 3) according to the second algorithm, The tables do not require any 
special explanation. 

In Tables 5.10 and 5.11 we give a solution of Example 1 (see Tables 5.2) following the procedure in which 
the vector to be introduced into the basis in each iteration is computed according to the rule given in 5-3, 
We see that the number of iterations in this modification of the second algorithm is less by one, The reduc- 
tion in the number of iterations, generally speaking, is not characteristic of this rule for selecting vectors to 
be introduced into the basis, Comparing the auxiliary tableaus in Tables 5.7 and 5,11 we see that the com- 
putations are less bulky. In Table 5.11 fewer evaluations A, had to be computed. 


6-2. We now compare the two simplex algorithms. Computations under 
the first algorithm involve only one kind of tableaus and are standardized 
to a higher degree than the computations under the second algorithm. In 
manual calculations this is a considerable advantage. Moreover, the first 
algorithm will establish unsolvability of the problem sooner than the second 
algorithm. As we shall see in § 8, the first algorithm is more convenient 
for solving the auxiliary problem in which the initial support program of 
the principal problem is determined. Finally, the first algorithm ensures 
that the vector A, introduced into the basis will produce the maximum pos- 
sible increase of the linear form in that iteration: 


—A, 00) = max (— Afi”). 


Computations under the second algorithm require less memory storage 
space than computations under the first algorithm. Under the first algo- 
rithm, the array corresponding to all the restraint vectors is stored. The 
volume of information stored under the second algorithm is determined 
only by the basis vectors. In the product form of the second algorithm the 
volume of data stored in each iteration is still less. 

In applied problems the percentage of zeros in the restraint matrix is 
generally very high. In the first algorithm, after elementary transforma- 
tion the zero components of the restraint vectors are converted, generally 
speaking, into nonzero components. Under the second algorithm, in which 
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the restraint matrix as a whole is not transformed from table to table, the 
relative advantage of the restraint vectors with numerous zero components 
is retained throughout the computations. In particular, the unit restraint 
vectors are very easily evaluated under the second algorithm: 


m 


When solving a problem following the second algorithm, the optimal pro- 
gram of the primal problem is obtained simultaneously with the optimal 
program of the dual problem (the vector A°=(A;,...,4°,), occupying the 
(m+ 1)-th row of the last principal tableau) and with the inverse matrix of 
the optimal basis. This feature of the second algorithm is most useful in 
some applications. 

When solving a problem according tothe second algorithm, we refer 
more often to the initial data than with the first algorithm. It is, therefore, 
obvious that computational errors in the second algorithm accumulate more 
Slowly than under the first. 

Let us now compare the two algorithms with regards to the number of 
multiplications and divisions per iteration. 

In 2-4 we showed that the solution of the linear-programming problem 
according tothe first algorithm requiresatmost n+1 divisions and m(n—m-+ 1) 
multiplications in each iteration. In 5-3 we established that each iteration 
in the second algorithm involves at most 2m-+1 divisions and m(n+m- }) 
multiplications. We see that the second algorithm, involving transformation 
of the inverse matrix and determination of the vector to be introduced into 
the basis from the least evaluation A,, involves, generally speaking, more 
operations than the first algorithm. 

The number of multiplications in each iteration of the second algorithm 
can be reduced if, as indicated in 5-3, the vector to be introduced in the 
basis is chosen independently of the computation of the A, of all the restraint 
vectors. In this case, however, the computational bulkiness of the two 
algorithms cannot be compared with respect to the number of operations per 
iteration, since the solution of a problem according to different algorithms 
will involve, generally speaking, a different number of iterations. 

In some cases computational bulkiness is somewhat reduced by the pro- 
duct form of the second algorithm. 

Observe that the comparative evaluation of the computational bulkiness 
of the first and the second algorithms in Wagner's paper /13/, which is 
quoted in certain works, cannot be considered objective. In this paper it 
is indicated that with n> 3m the first algorithm invariably involves bulkier 
computations than the second algorithm. When determining the number of 
multiplications performed under the first algorithm, Wagner assumes 
that the evaluations A, are computed in the first algorithm not from the 
recurrence formulas (2.2), but directly from (5.1). Wagner's conclusion 
would apply if it were found necessary to check computations after each 
iteration. In Table 5.12 we give the number of multiplications required for 
a single iteration and for control purposes in the first and the second sim- 
plex algorithms. We see from the table that for n> 3m the first algorithm 
is bulkier than the second, It is, however, quite unnecessary to check the 
computations in each step of the method. 

This comparison of the two algorithms does not take into consideration 
the specific features of particular classes of linear-programming problems. 
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In problems with a high percentage of zeros in the restraint matrix (these 
problems are quite frequent in applications), the number of operations per 
iteration under the second algorithm is less than under the first. We have 
already seen that the most tedious stage of computations under the second 
algorithm — determination of the evaluations A,- is essentially reduced 
for restraint vectors containing numerous zeros. 


TABLE 5.12 


1st algorithm 


Per iteration | m(n—m +1) | m(n+m-+1) 


Check m (n+1) | m(m+t) 


2nd algorithm 


Total | m (2n—m-+-2) | m (2m +n-+ 2) 


Let each column of the restraint matrix contain at most am nonzero ele- 
ments, The computation of A, involves am(a—m) multiplications. The 
transformation of the principal part of the tableau involves, as before, 
m(m+1) multiplications, and the elements of the direction column are com- 
puted in A!) —m* multiplications. The total number of multiplications in 


each iteration in the second algorithm are, thus, at most 
am (na —m) + m(m+1)+m’ =m [an + (2—a) m+ 1). 


The bulkiness of computations under the first algorithm is independent of 
the number of zeros in the initial restraint matrix. The total number of 
multiplications in each iteration in the first algorithm is m(n—m-+ 1). 
Thus, when 
m [an + (2—a) m+ 1] < m(a—m-+1), 


i.e., when 


3—a 
a> T=a ™ 


the second algorithm involves less multiplications than the first. 
Analogous evaluations can be made for other particular classes of 
linear-programming problems. 
The comparison of the two algorithms discussed inthis section shows 
that one algorithm may be preferable to the other depending on the problem 
under consideration. 


§ 7. The case of bilateral restraints 


7-1. In Chapter 4, §5, we showed that when linear-programming prob- 
lems are subject to bilateral restraints, the simplex computational pro- 
cedure remains essentially the same and only some elements of each itera- 
tion become more complicated. The modifications of the algorithm per- 
tain, mainly,s to the method of determining which vector is to be 
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eliminated from the basis, the transformation formulas of the basis vari- 
ables (the coefficients in the expansion of the vector 


J ty 


and the evaluations A, of the restraint vectors. 

Consider the modified structure of the tableaus for a problem with bi- 
lateral restraints solved by the first algorithm. The /-th tableau corres- 
ponding to the /-th iteration of the problem with bilaterally restrained 
variables differs from the /-th tableau compiled in the solution process of 
the problem incanonical form in that there are three additional rows and two 
additional columns (see Table 5.13). In the row (a, Bf) of this table, above 
each restraint vector not included in the basis, we write the value of the 
corresponding extrabasis variable. If x, is equal to its lowest boundary 
value, this should be signified by the letter a, and if x, is equal to the 
highest boundary value, this should be signified by the letter §. The entries 
in the (a, 8B) row corresponding to the basis vectors are crossed out. 


TABLE 5,13 
t-th tableau 


_ — en —1;V¥t! (é) t 
eee I eee « 1 Vv Ym +] gf!) 


a | es ee, ey 


m+2,1 *m+2,2 a e e m+ Qsy e e eo 


The evaluations AW, which in the case of unilateral restraints (see § 3) 
were written in the (m+1)-th row, are written in the case of bilateral re- 
straints in the (m+3)-th row of the tableau. In this case 

Ap ma, 5 A, = L(X), 
The elements written in the (m+ 1)-th and (m+2)-th rows will be specified 
below. 

To the right of the columns corresponding to the restraint vectors we 
have two new columns, A, and y. The column A,contains the coefficients 
in the expansion of the vector A, in terms of the basis vectors, written 
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with their respective sign if *,=@ and with reverse signs if x,=6,(A, is 
the vector to be introduced into the basis). In other words, the elements 
of the column A, are equal to 


xin =(—1)"sfe, 
where y=0for x,=a, and y=1 for x,=f,. 


In the next column, yY, we write a, if the corresponding position of the 
column A, is occupied by a positive number, and 6, if xQ<0, i.e., 


s6 (1) 

Aes a (7.1) 

iv ik <0. 
The entries in the column y for which y{ =—oo or yj =oo and likewise the 
positions corresponding to xf? =0 are crossed out. 
Each entry in the last column 6 is computed as the difference of the cor- 

responding elements in columns A, and y divided by the element of the 
column A, in the same row: 


Wo 
Q() a 70 v ; (7.2) 


In the (m+ 1)-th row of the table only the five positions corresponding to the 
columns A,, A, 4A,, y, and @ are filled. The (m+1)-th entry in the column A, 
is the parameter x‘?.,,.=+t? (a,or B,), i.e., the value of the extrabasis 
variable corresponding to the vector A, introduced into the basis. The 
(m+1)-th entry in column A, is always the number x%.,,.=—1. The values 
of x... Yes, and §,,, are computed according to the general rules for com- 


puting the entries of the columns Af}, y, and 0: 


~( | 
Neves, k= ee: k(— 1)’ = (— Ly", 


‘ t) 
ve =| Ay, if x =B,, 


Pd 
6 = mest m+ _ =3 ; 
m+ gD ok B, a, 
In the (m+2)-th row we write the parameters Kinsey (j=0,1,. .,m) required for 


computing the evaluations Aj). Formulas (7.4) for the computation of Om 


are given in 7-2, 

In the case of problems with bilateral restraints, the principal part of 
the i-th tableau in the first algorithm comprises all the entries of columns 
A,, A, .-., A, with the exception of the (m+ 1)-th and the (m+-3)-th rows. 

As in the general case, the form of the initial zeroeth tableau differs 
from the form of the /-th tableau in the additional row C containing the li- 
near-form coefficients ¢, The entries of the zeroeth tableau are not com- 
puted from recurrence formulas. The parameters x,;, are computed from 
systems of equations (see (5.5) in Chapter 4), and the evaluations A, are 
obtained from (5.6) and (5.7) in Chapter 4. Using any of the methods of 
determining the initial support program it is possible to obtain at the same 
time the corresponding values of the coefficients x,;, and the evaluations A,. 

7-2, We now outline the sequence of computations to be followed in the 
solution of a linear-programming problem with bilateral restraints accord- 
ing to the first simplex algorithm. 

Assume that the /-th iteration has been completed. Thus, the principal 
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part of the /-th tableau and the (m+ 3}th row are filled. The computations 
in the (/+ 1)-th iteration will fill the (m+ 1)-th row, the columns Ay, y, and 6 
of the /[-th tableau, and the principal part of the (/+1)-th tableau. In the 
first stage of the (/-+1})th iteration the (m+3)-th row of the /-th tableau is 
examined. If all the Af?=0, case (a) applies and the support program ob- 
tained in the [-th iteration solves the problem. If there are vectors with 
negative evaluations (A,<0), case (b) or case (c) may apply. Case (b) applies 
if, for some j, 

(i) when x,=a,, for x,,>0 a,=—oo, and for x,,<0B,,=oo, or 

(ii) when x,=6,, for x,,>0 B, =o0o, and for x,;<0a,=—oo, We shall refer 
to (i) and (ii) as the unsolvability criteria of a problem with bi- 
lateral restraints. 

Case (b) indicates unboundedness of the linear form in the set of feasible 
programs of the problem. 

If unsolvability has not been established (i.e., if case (c) applies), we 
must proceed with the second stage of the iteration. In the second stage, 
the following operations are carried out: 

(1) the vector to be introduced into the basis is determined; 

(2) the vector to be eliminated from the basis is determined; 

(3) the coefficients xf) in the expansion of the restraint vectors in terms 
of the basis vectors (4=1,2, ...,m; /=1,2,...,a) are transformed; 

(4) the basis variables x? and the value x".:,0 of the linear form are 
transformed; 

(5) the row (a, §) is transformed; 

(6) the evaluations A‘ of the restraint vectors are transformed. 

We consider each of these operations separately. 

(1) The vector A, with the least evaluation Aj? is to be introduced into 
the basis. Having chosen A,, we fill the columns A,, y, and @ and the 
(m+-1)-th row of the /-th tableau. 

(2) The vector A, for which 


0, = min” 
j 


is to be eliminated from the basis. If the program is degenerate, 0!) is 
obtained on several vectors, the vector with the least index is eliminated. 
When cycling occurs, the more complicated rule described in 7-4 is applied. 

(3) The coefficients x!) (i=1, ..., m; j=1, ..., a)in the expansion of the 
restraint vectors in terms of the basis vectors are transformed according 
to the recurrence formulas (1.12) if 6” is obtained on one of the first m 
positions of the basis. If 6 is obtained on the (m+1}th position of the basis, 
the basis remains as it was and the coefficients x,, retain their original 
value, i.e., x4* = xf) for r=m+1 (f=1, 2, ..., m fal, 2, ..., A). 

(4) The basis variables are transformed according to the recurrence 
formulas 


rit | xo 9s wheni#r, (7.3) 
; a 41,005) (— 1)’* 'wheni=r, 
lagliegm. 


Formulas (7.3) apply regardless of whether 06) is obtained on the first 
m positions or on the (m+1)-th position of the basis. The same recurrence 
formulas, (7.3), are used to transform the value a % of the linear form. 
To this end the evaluation A of the vector to be introduced into the basis 


is written in the (m+2)-th cell of the column (x!,,, »= Af’). 


219 


(5) In the row (a, §), at most two entries are modified. For ram the 
vector A, is introduced into the basis and, consequently, the entry in the 
row (a, 8) corresponding to this vector is crossed out. In the position cor- 
responding to the vector A, eliminated from the basis we write the bound- 
ary value assumed by the component <x«,,: 


bon 4 4) 7) (4) 
xa," == xre — ON) Xrk = Yr 


For r=m+41 the basis remains unmodified. Therefore, the position cor- 
responding to the vector A, inthe row (a, 8) remains crossed out, and the 
boundary value of the component +x, in the row (a, B)is replaced by the op- 
posite member of the pair. 

(6) The relationship between A{/*” and Af? depends on whether the basis 
changes upon transition to the new program. 


| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
| 
! 


-——— 


@ | Test A,<0 | no j. Case (a) 

a a optimal program) 
: yes | 

~~ 

Be SF Ans Case (b) 


~~ 


unsolvable problem) 


ume he ee ee ee es ee ee ee ee ee ee ee ee 
~~; 


[-------~---------- 


Test for unsolvability 
| 


2nd stage 


Transform the principal 
part of the tableau by re- 
currence formulas 


Fill the rows 
(a, B) and A, 


FIGURE 6.3 
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First letr<m, Further, let 


t ! = I 
Xmie, |= 2] —Cy= DY Cyt; — Cy j=1, 2, ..., 2. (7.4) 


Obviously, 
AY? = (1), p= xO, I J=1, 2, ..., 4, (7.5) 
where 


v,=0, if x,=a,andv,=1 for x,=6,. 
The parameters Pla ;are written in the (m+ 2)-th row of the tableau and 

are transformed according to the same recurrence formulas (1.12) as the 
coefficients x‘? in the expansion of the restraint vectors in terms of the 
basis vectors. Having transformed the parameters Seca: , according to (7.5), 
we compute the transformed evaluations. 

If r=m+1, the (m+2)-th row remains unmodified, except for the cell in 
which the value of the linear form is written. For r=m+1 the basis re- 
mains the same and the evaluations of all the restraint vectors, excepting 
A,, are retained. The sign of the evaluation of the vector A, is reversed. 
Thus, for r=m+1 

afranf af for s+ 
— Ai for j=k. (7.6) 


Having computed these parameters, the (/+1)-th iteration terminates and 
the principal part of the (/+1)-th tableau can be filled. Subsequent iterations 
follow the same rules. 

In Figure 5.3 we give a block diagram of a single iteration in the solu- 
tion of a linear-programming problem with bilateral restraints according 
to the first simplex algorithm. 

7-3. We now outline the sequence of computations for the solution of a 
problem with bilateral restraints according to the second algorithm. 

For the problem with bilateral restraints the structure of the principal 
and the auxiliary tableaus, which were compiled in § 5 for the linear-pro- 
gramming problem in canonical form, are slightly modified. 

In the principal tableaus we add (as in the first algorithm) two columns, 
A, and y. The columns are filled following the same rules as in the first 
algorithm. 

The evaluations of problem restraints Af (s=1, 2,..., m)and 4,=L(X)are 
written, not in the (m+ 1)-th row, as in the case of problems in canonical 
form, but rather in the (m+2)-th row. The (m+ 1)-th row is filled with the 
same parameters as the (m+-1)-th row in the first algorithm. Five cells are 
filled in the (m+1)-th row: 


(2) (t) ~() ) 
Em+t,0— Xpgy Xmes,h=—l, Xmere = (—1) or 


I t 
Vo es and 0 = 6, —d,. 


After each iteration not one row, as in §5, but two rows, (a, Bp) and A‘, 
are added to the auxiliary tableau. In the upper part of the auxiliary tableau 
it is advisable to introduce the row a,/6, which gives the lower and the upper 
boundary values of the corresponding variables. When the /-th iteration is 
completed, the principal part of the /-th principal tableau and the /-th pair 
of rows in the lower part of the auxiliary tableau are filled. 
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The vector A, to be introduced into the basis is determined as in the 
first algorithm. The coefficients in the expansion of the vector A, in terms 
of the basis vectors and the vector A, to be eliminated from the basis are 
determined in the same way as in the second algorithm for problems in 
canonical form. However, unlike the case of canonical form, the possibi- 
lity r=m-4-1 may arise. 

The coefficients e) of the unit vectors when expanded in terms of the 
basis vectors (i=1, 2, ..., m; j/=1, 2, ..., m)are transformed according to re- 
currence formulas (1.21) if 6 is obtained for r<m, and remain unmodified 


ifr=m-+1. 


| Test A,<0 | she i_, Case (a) 
|(optimal program) 
yes | 


Choose vector A, and 
calculate x, 


H 
; 
Test for unsolvability Pe hii ee Case (b) 
| (unsolvable problem) 
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currence formulas 


FIGURE 5.4 


The basis components of the program, i.e., the entries of the column 


e, sain? > x,A, are transformed in the same way as the entries of the column 
f 
x 
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A, in the first algorithm: 


aia{ RaW for ter 
ee — 00 (—1)"*" for iar, (7.7) 
|? oe Pore 


Formulas (7.7) apply regardless of whether the basis is modified upon 
transition to the new program or not. The same recurrence formulas are 
used to transform the value Oe ei of the linear form. To this end, in the 


(m+2)-th position of the column A, we write the evaluations 


~i( t 
Leia a= A} 


of the vector introduced into the basis. 

The rows qa, f) of the auxiliary tableau are transformed following the 
same rules as in the first algorithm. The parameters Aj” are computed 
from recurrence formulas (1.21) with jem+1, j=1, 2, ..., a. 

The parameters A!) are computed from the formulas 


>» a,aA}—c, for x,=G@,, 


e— Saal” for x,=B,. 


To unify the procedure we suggest that the difference Sa,d—c, be com- 
=) 


Af? = (7.8) 


puted without taking into account the actual values of the extrabasis varia- 
bles, and then, when examining the row (a, B), the signs of the entries for 
which x,=68, should be changed. 

Figure 5.4 shows a block diagram of the second algorithm for a problem. 
with bilateral restraints. 

7-4. We now deal with the specific features of the procedure for chosing 
the vector to be eliminated from the basis inthe case of degenerate prob- 
lems with bilateral restraints. 

When solving the problem according to the first algorithm, it is advis- 
able to define R, thus: 

A, for 2x,<6s,, 
R= yi for oe (7.9) 


When the second algorithm is applied, the following system of vectors R;, 
should be used: 
R=AyW; R =e, ..., Ryze), 
Rey yy veer Ramey, (7.10) 


Here e, are the m-dimensional unit vectors; the components of the vec- 
tor W are determined by the formula 


1 for %in<Bs,; 
v= —1 for Xi, = Bs, 


Ay is the basis matrix, and the index f is chosen so that the ¢-th coefficient 
of the vector A,W is nonzero. 

It can be easily verified that the system (7.9), and likewise (7.10), satis- 
fy the two conditions imposed on R, in §6 of the preceding chapter. 

7-5. We shall now illustrate the sequence of computations leading to an optimal program of a linear- 
programming problem with bilateral restraints. 

Maximize the linear form 


L (X) we Xq Deeg + 3k y + q+ BH + Ig + Sy — Fy —— XQ — Rpg X4y 
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subject to the conditions 


Ry X22, +3x,-+- 2x,+3x,-+ x, +2, =7, 
2k, +3x%g+ Xy+ Xt 3x,+2e, +2, +2, =8, 
X, +22, + 3x, + 2x, +0, 5x,5-+ Xe+ +X, =6, 
244+ %+3x,-+ %+ 2x, + 3x, + x, +21, =7, 


O<xql, j=l, 2, eas 7, 
xj=20, J=8, ..., Ll. 


The entire solution process of the problem by the second simplex algorithm is summarized in Tables 5.14 
(the principal tableaus) and in Table 5,15 (the auxiliary tableau). 

The unit basis (Ag, As, Aye, Ay) is, obviously, taken as the initial basis, The corresponding support 
program has basis components equal to the components of the restraint vector, and zero extrabasis components. 
The column @é, of the principal zeroeth tableau, in which the components of the vector 


A= B— Bi APs 


are written will, therefore, only contain components of the vector B. 

The first m=4 entries of the columns @¢, ..., @, of the zeroeth tableau constitute a unit matrix. The 
(m-+-1)-th, Sth, row is filled after the vector to be introduced into the basis has been determined. The 
(m-+2)-th, 6th, row contains the evaluations A;of the problem restraints and the value of the linear form. 
Thus, 

L (X) =A, =— 1-7—1-8—1-6—1-7 = — 28. 


The other A,({=1, ..., 4) are equal to the linear-form coefficients associated with the corresponding basis 
variables, 

In the row (a, B) of the lower part of the auxiliary tableau corresponding to the principal zeroeth tableau 
we indicate that in the initial support program all the extrabasis variables assume their lowest boundary value 
(a,) (when filling the auxiliary tableau, we need indicate only which of the boundary values the extrabasis _ 
variables assume; the boundary value is not used in determining the sign of Ay). In our case, all the extra- 
basis variables vanish (being equal to the lowest boundary value), and the evaluations A, are computed from 
the first of equations (7.8). The least Ay(—12) is obtained on the vectors A, and A,. Either of these vectors 
can be introduced into the basis. 

Let us choose the vector A. In column Ag, of the principal zeroeth tableau we write the coefficients 
in the expansion of A, in terms of the basis vectors: 


Xm+i,a=%s,s=— 1, X42, z= A= — 12. 


The column Ay coincides with the column Ag, since all the extrabasis variables assume their lowest boundary 
values. The first m==4 positions of the column y are filled with zeros, The parameter 


Yn+1=Y¥s=B8,=1, 


since ¥,;=a,. The 6-values are computed from (7.2). For example, 


6, = Xso— Yu 8 —0__g 
Xe l 


Xs9—- 0—1 
Cass =H = 
26 


=}, 
In our case 0,,.,,= 6, is the least number in the column @. Therefore, when passing to the next 1-th tableau, 
the basis remains unchanged. 

The variable x, corresponding to the vector A, assumes, on the new program, its highest boundary value 
(and not the lowest as assumed in the previous program). The sign of A, will change accordingly. All the 
other extrabasis variables retain their original values, and the corresponding vectors retain their respective 
evaluations, We see from the auxiliary tableau that now the least evaluation corresponds to the vector 
A, (A, =— 12). The vector A, is introduced into the basis, 

The column é, of the principal l-th tableau contains the components of the vector 


+ 
A,=B— & x,Ap 
In our case x 


e- = B—B,A,=B—A,. 
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Thus, when passing from the zeroeth tableau to the 1st tableau the entire principal part of the tableau, ex- 
cept for the column A, (entries 1,2,..., m, m+2), remains uncharged, The columns Ag, Ag y, and 6 
and the (m-+1)-th row are filled as in the preceding tableau. The minimum is again obtained in the 
(m+1)-th entry. Hence, in this iteration, too, transformation to a new support program will not affect the 
basis. 

In the next iteration (2nd tableau) the vector A, is introduced into the basis in place of the unit vector 
é;j. In this case the principal part of the tableau is transformed according to the recurrence formulas (the 
basis components of the support program and the value of the linear form are computed from (7.7), and the 
elements A,, of the inverse matrix and the relative evaluations A, of the problem restraints from (1.21)). 

After six iterations, all the evaluations Ay of the restraint vectors become nonnegative. The optimal 
basis comprises the vectors Ay, Ag Ag, Ag, The basis components of the solution are 


%,=20.764, x,==0.382, x, 0.982, x, ==0.455. 


Of all the extrabasis variables only x,=1 is nonzero, 


§ 8. Computational procedure for determining the 
support program 
ze 

8-1. In discussing the simplex algorithms, we invariably assumed the 
initial support program to be known. 

In §7 of the preceding chapter we gave some theoretical considerations 
on the possibility of constructing an initial support program and described 
procedures for combining two stages, namely computation of the initial 
program and solution of the problem. In the present chapter we will deal 
with the realization of the corresponding computational procedures. 

First, consider a method of computing the initial support program which 
amounts to solving the so-called auxiliary problem (see Chapter 4, §7). 

To solve the auxiliary problem, we may, generally speaking, apply either 
the first or the second algorithm. In some cases, however, application 

of the first algorithm entails less computations. The reason for this is 
that the optimal basis of the auxiliary problem often ccntains, besides re- 
straint vectors of the principal problem, also the unit vectors e, (artificial 
vectors). To replace the unit vectors by restraint vectors, we must know 
the coefficients in the expansion of all the vectors A, in terms of respective 
basis vectors. The coefficients x,, for all / are computed according to the 
first algorithm only. 

When solving the auxiliary problem, we need not compute the coefficients 
in the expansion of the unit vectors in terms of the basis vectors. The unit 
vectors constitute the initial basis of the auxiliary problem. However, 
when solving the problem, it is not advisable to include artificial vectors 
in the basis. Therefore, we may omit from the tableau of the algorithm 
all the columns corresponding to the unit restraint vectors of the auxiliary 
problem. 

The process of solution of the auxiliary problem, whose initial program 
is obvious, terminates after a finite number of iterations in case (a) (case 
(b) is ruled out, since the linear form of the auxiliary problem is zero- 
bounded above). Two possibilities may arise. If the maximum value of the 
linear form is zero, the basis variables of the auxiliary problem specify 
the initial support program of the principal problem. If not all the artifi- 
cial variables vanish in the solution of the auxiliary problem, the principal 
problem is unsolvable. 

Let the optimal value of the linear form of the auxiliary problem be zero. 
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As we have already indicated, the optimal basis may contain additional 

unit vectors. The corresponding basis variables all, obviously, vanish. 

We shall show, without modifying the program, how to replace the unitbasis 
vectors by restraint vectors, thus complementing the basis to the maxi- 
mum system of linearly independent restraint vectors of the principal prob- 
lem. If the maximum system contains m vectors, the rank of the restraint 
system of the problem is also m. If the maximum system contains r<m 
vectors, then, by suitable transformation of the basis, we shall isolate r 
linearly independent equalities — problem restraints. 

Consider the tableau corresponding to the last iteration of the solution 
process of the auxiliary problem. Two cases are possible: 

(a) allthe elements «x,, of the principal part of the tableau in all the rows 
corresponding to the additional unit vectors are zero; 

(b) the optimal basis of the auxiliary problem contains an additional 
vector e, for which at least one of the coefficients +x,; is not zero. 

In (a) the restraint vectors of the principal problem entering the optimal 
basis of the auxiliary problem constitute the maximum system of linearly 
independent vectors of the matrix A=|ja,,||,,,, Indeed, these vectors belong 
to the basis and are, therefore, linearly independent. Moreover, any 
restraint vector can be expanded in terms of the basis vectors so that the 
coefficients in the expansion corresponding to the additional unit vectors 
are zero. Hence, all the vectors A, can be expanded in terms of the re- 
straint vectors of the principal problem entering the optimal basis of the 
auxiliary problem. 

We now consider (b). Let the é#th position of the basis of the auxiliary 
problem contain an additional unit vector e, and let x,,#0. We substitute 
restraint vector A, for the basis vector e; The resulting system of vec- 
tors remains linearly independent and, as we see from recurrence formu- 
las (1.12), the elements of column A, retain their previous values. The 
number of additional unit vectors in the basis of the optimal program of the 
auxiliary problem is thus reduced by one, whereas the program is not modi- 
fied in the procedure. Similarly, the restraint vectors of the principal 
problem can be successively substituted for all the additional unit vectors 
to each of which correspond at least one nonzero coefficient x,; The 
maximum system of restraint vectors will contain m vectors, if all the 
additional unit vectors are removed from the basis of the optimal program 
of the auxiliary problem. The maximum system contains r<m vectors, if 
after substitution we obtain a tableau in which all the elements of the prin- 
cipal part corresponding to the m—r unit basis vectors are zero. In such 
a case, the tableau rows corresponding to the unit artificial vectors should 
be omitted. The construction of an optimal program of the principal prob- 
lem is analyzed with the aid of an rxa restraint matrix. Thus, in the pro- 
cess of solution of the auxiliary problem, an initial support program is 
determined and the independent restraints of the principal linear-program- 
ming problem are isolated. 

8-2, We shall illustrate the above by a simple example, 

Determine the initial support program for the problem of maximization of some linear form L (X) subject 
to the conditions 


2%, +3x,—- X,+ 44+ x,=> 8, 
3x, + 2%, +2%,— 2x,—2x,= 7, 
5x, +5x,-+ Xe M— X= ld, 
%, + 42,—4x, + llz,4+4x,= 9, 
xjon0, Jarl, 2,..., 5. 


231 


TABLE 5.16 (0-2) 


“TaD [s 


marc 
tare 
a ote fon 


1 


N 
a 
=“ 
wv 
w 


232 


The auxiliary problem involves maximization of the linear form 


L (X)=— (x, +%, +2, +4%,) 


subject to the conditions 


L, (X) +4, = 8, 
LAX) +x, =7, 
L; (X) +%, = 15, 
L, (X) +x, =9, 


x, 0, j=1, 2: eoee 9. 


Here L;(X) are the left-hand sides of the restraints of the initial problem, We shall solve the auxiliary 
problem applying the first simplex algorithm, The solution process is obvious from Table 5.16, As pointed 
out it is unnecessary to include columns in the tableaus for the artificial vectors. 

The optimal basis of the auxiliary problem contains two artificial vectors A, and A,. We see from the 
2nd tableau that the coefficient x) in the expansion of the vector A, in terms of the vectors of the last basis 
is nonzero, The artificial basis vector A, can, therefore, be replaced by the restraint vector Ay Having 
transformed the last tableau, we obtain Table 5.17, The elements of the row A, are all zeros, Hence, the 
vectors A,, A,, and A, constitute the maximum system of linearly independent vectors of the restraint 
matrix, 

The optimal value of the linear form of the auxiliary problem is zero. Therefore, the solution of the 
auxiliary problem specifies ‘the initial support program of the principal problem. Moreover, we see from the 
last tableau that the linearly independent restraints of the principal problem can be rewritten in the form 


5 5 
B* +x, —“%=F» 
X, =0, 
7 23 
Butts =-3° 


Thus, solving the auxiliary problem we obtain not only an initial program of the problem, X= (0, , 


5 0,0) 


T° » but also a simplified form of the problem restraints. 

8-3. We will now see how to use the M-method which makes it possible 
to combine the determination of the initial support program with the solu- 
tion of the linear-programming problem. The optimal program of the M- 
problem, whose initial program is obvious, can be computed either accord- 
ing to the first or to the second simplex algorithm. 

The linear-form coefficients ¢, of the M-problem are linear functions 


of the parameter M: 

C,= aM+e, 
The parameters A, and 4, are, therefore, also linear functions of M: 
Hence the specific features of the tableaus in the computational procedure 
of the M-method. Solving the M-problem, according to the first or the 
second algorithm, not only one row, A, in the tableaus is filled but two 


rows, the A row of the coefficients of Mand the A of the free terms. The 
evaluations A,of the vectors A, are compared as follows: 


Ay, > Ay: 
A,=A ? but A, 


A, ? Ay» if >A,, 

Correspondingly, the evaluation A, is considered positive if either A, >0 
or A,=0, but A,>0. 

Computing according to the second algorithm, the last row of the prin- 
cipal tableaus (the row of the j-evaluations of the problem restraints with 
respect to the current basis) is also replaced by two rows, 4 and A. 

The artificial vectors eliminated from the basis are not considered in 
the solution of the M-problem. The additional unit vectors are, therefore, 
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not included in the columns of the restraint vector inthe tableaus in which 
the solution of the M-problem is written. It can easily be seen that this 
rule reduces the volume of computations, without essentially affecting the 
M-method. 

The solution process of the M-problem terminates, after a finite number 
of iterations, in case (a) or case (b). In case (a) the solution of the M- 
problem specifies an optimal program of the principal problem, if the addi- 
tional variables do not enter the solution. All other possibilities mean un- 
solvability of the principal problem (the linear form is unbounded in the 
set of feasible programs of the problem or the problem restraints are in- 
consistent). 

Apart from the above special features, the process of solution of the 
M-problem according to the first or second simplex algorithms is the same 
as that discussed in §2 and § 5. 

8-4, We now illustrate the application of the M-method to the solution of a linear- programming prob- 
lem whose initial program is not obvious, The computations will be carried out according to the second 
simplex algorithm, 

Maximize the linear form 


: ae L (X) = 2, + 2x, + 3x5 +2, + 2%, + 3x, + x, (8.1) 
subject to the conditions 


X,4+ %+2,+3x,+ 2e,+3x,+ x,=7, 
2x, 4-3x,+ Xy3+ X+ 3x4 2e,+ 2x, =—8, 
X, + 2x, 43x, +2x,4+0,5x,+ 2x,4+2x,=6. 
Qe, Xyt3x,+ Xt 2x, +3x,+2x,=7, 


xj20, J=1,2,...,7. 


(8, 2) 


No initial support program is given. We introduce artificial nonnegative variables 2X, %,, X49, and x, 
and consider the corresponding M- problem. 
Maximize the linear form 


C=L (X)— M (X%y + %y + X49 +211) 


subject to the conditions 


L,(X) +4, =7, 
L, (X) +, = 8, 
L, (X) +X =6, 
L, (X) +X =7, 


xj20, j=1,2..., UL. 


Here L (X) is the linear form (8.1) and L;(X) represents the left-hand sides of restraints (8,2) of the principal 
problem. 

We give here the principal and the auxiliary tableaus (Tables 5.18 and 5.19) which illustrate the succes- 
sive program improvement according to the second algorithm, 

The tableaus do not need special explanations, The solution is obtained in four iterations, The optimal 
program is 


The maximum value of the linear form is 137/16. 

In this example the optimal program is obtained directly when all the artificial vectors have been elimi-~ 
nated from the basis. In the general case, however, this need not be so, After the artificial vectors are 
eliminated from the basis, we usually obtain a support program which is then transformed by the simplex 
procedure into an optimal program, 

8-5, We shall now show how computation of the initial support program 
can be simplified in certain cases. 

Until now we associated the auxiliary problem andthe M-problem with 
a complete artificial basis, namely with m additional unit vectors. The 
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TABLE 5,18 (0-4) 
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TABLE 5,19 
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bulkiness of computations is considerably reduced if the principal problem 
has some unit restraint vectors. In several cases simple manipulations 
enable us to reduce the number of artificial variables of the auxiliary prob- 
lem or the M-problem. 

Let the problem restraints be given in the form 


> a,j, & b,, 


{=1 
i=1,2, ...,m; %,20, J=l,2,...,8 6,20. 

As we have remarked (Chapter 5, §7), the additional nonnegative varia- 
bles x,,, (j=1, 2, ...,m) reduce the problem to canonical form. The values 
of the additional variables +*,,,=6, specify the initial support program. 

Now let the problem restraints have the form 


A 
tuts? bp 
d=1,2,...,m; x,0, j=l,2,...,a 5b;0. 
The additional variables x,,,; (é=1, 2, ...,a) will again enable us to reduce 


the problem to canonical form. However, to obtain the initial support 
program we must, in this case, solve an auxiliary problem (or the #prob- 
lem) with one artificial vector. We have 


an 
2 Gj JX ,—X qa = bj, 


i= 1, 2, oon, Mm, x, 0, j= 1, 2, coe A+M, 
Let b,=maxb, We transform the system of restraints to the form 
a 
Pa(aa— 41) Xp Kags t Xn (= Os—Op i#xs, 
nl 
Pa te*s—Fnse= bn 4,20, jJ=i1, 2, ee ATM, 


Thus, a problem where m—1 positive unit restraint vectors are associated 

with the nonnegative components of the constraint vector is obtained. De- 

termination of the initial support program thus involves one artificial vector. 
Finally, let the problem restraints be given in canonical form: 


n 


Pu sikj= Op dus, 2,...,m, X20, fl, 2,..., 2. 


Let all 6,=0 but at least one of the components of the constraint vector be 
positive. 

There are three possibilities: 

(a) there are no unit vectors among the restraint vectors; in this case 
the M-problem (or the auxiliary problem) contains m artificial variables; 

(b) the matrix || a,,\| contains r different unit restraint vectors; in this 
case the M-problem contains m—r artificial vectors. 

(c) the restraint matrix ||¢,,{)contains a unit submatrix of order m; the 
initial program here is obvious and there is no need to resort to the M- 
method or to solve the auxiliary problem. 

In some linear-programming problems a system of linearly independent 
vectors can be isolated from among the restraint vectors. In these cases 
determination of the initial support program is greatly simplified. Let the 
vectors A,, A,,..., A, be linearly independent. We solve the system of linear 


equations 


Pa suti= or i=l, 2, coey M, 
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for the variables x,,..., x,. We have 
a 


X,=XE+ D2 XtjXp 
fami 


where xf and xy, (i=1,...,m;j=1,...,”)are constants. If all x{=0, then, 
taking x,=0 for j=m+1,...,a, we obtain a support program X = (X}, 006 Sms 
O:.-heay 0), 

Let now some of the xf be negative, x, being the least. We subtract, 
term-by-term, the s-th equation from all the equations of the system. Then 


R n 

x, =($— x) +x, t+ (xt — atx xy QD xix, 
f=m+i fam+1 

This transformation yields a basis comprising m—1 positive unit restraint 

vectors of the problem and one artificial vector. 


§ 9. Cycling in linear-programming problems 


9-1, In Chapter 4, §6 we gave rules for determining the vector to be 
eliminated from the basis which guarantee that cycling will not arise when 
solving linear-programming problems. As we have seen, in §§ 2, 5, these 
rules involve additional computational work. 

It is, therefore, natural that we should consider the possibility of cycling 
arising when a linear-programming problem is solved by the simplex 
method and the probability of this phenomenon. The present section deals 
with the conditions under which cycling arises; this discussion will provide 
the answers to the above queries. All reasoning refers to the problem in 
canonical form. 

For the following we require recurrence formulas relating the parame- 
ters of two successive simplex iterations. The suitable formulas were 
derived in §1. We repeat them here assuming that in the successive itera- 
tion the vector A, is introduced into the r-th position in the basis: 


Xip F 
a mp Oe Ee (9.1) 
xj 7 
Xen’ if i=r, 
$=1,2,...,m, /=0,1,2,...,a, 
, A 
A; =4,;—yF Xpp j= 1 2, 00 0, A, (9.2) 


Here, as elsewhere, the primed parameters refer to the successive itera- 
tion. 

Applying formulas (9.1) and (9.2) twice, we can establish a relationship 
between the parameters for any two iterations with one iteration between 
them. Assume that in the second iteration the vector A, is introduced into 
the r-th position, and in the third iteration the vector A, is introduced into 
ther,-th position. Then 


° Xp kh Xrj— rk Xp 7 


= ht et ae . 
Xi Xr hk Xrh— Xp kerk, ) i r#T,, i=re=l, 2, saacy ae 
J=90, 1, 2, eee A, (9.3) 
. 4 Xppna~ Xpy X, Xen Xp s——X x 
AT AA, rey Heh hr Prk Arf | 
_ - *, h, *rh—*rh *rk, i me k, Xreh—*r h Xeh, ye * uss (9 .4) 
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The double-primed parameters refer to the last iteration (of the three 
iterations in question). 

Formulas (9.3) and (9.4) apply in cases when the vectors A, and A, are 
introduced into different basis positions, and in (9.3) the i-th position is 
assumed to coincide with one of them. In this section we shall consider this 
case only. Derivation of formulas (9.3) and (9.4) are left to the reader 
(see Exercise 11). Formula (9.3) can, obviously, be adapted to the case 
i=r,#r. It suffices to substitute k, and r, for k and r, respectively. 

9-2, Assume that the rule given in Chapter 4, §6 is not applied when 
determining which vector is to be eliminated from the basis. Then, the 
choice of the basis position in which a new vector is to be introduced is 
equally simple in the nondegenerate and the degenerate case. Indeed, we 
choose the position r such that 


= min (9.5) 

Here & is the index of the restraint vector introduced into the basis, A,. 
We recall that the vector A, introduced into the basis has a negative evalua- 
tion A, with respect to the original basis. In the degenerate case there are 
several basis positions satisfying condition (9.5); any of these can be cho- 
senas r. We shall say that a vector A, is suitable for introduction into 
the r-th position in the basis if A,<0 and condition (9.5) is satisfied. 

We will study the possibility of cycling, i.e., the possibility of the 
following sequence of bases arising when solving a problem by the simplex 
procedure: 

B—B,—B,—...—>B,.,—~B (9.6) 

We shall say that a cycle isoflength d, if cycling comprises d 
iterations. The length of cycle (9.6) is thus ¢. 

Obviously, transferring from basis to basis as in (9.6) should not in- 
crease the linear form of the problem, since otherwise thetransition from 
B,., to B would be ruled out (the linear form cannot decrease'). There- 
fore, in all the iterations constituting cycle (9.6) all 6, are zero. 

Hence, in particular, all the elements of (9.6) are bases of the same 
support program. 

Let the basis B,,, be obtained from B, by introducing vector A, into the 
r-th position. Then, in order to transfer from B, to B,,, it is required that 
the vector A, be suitable for introduction into the r-th position. In this 


case 6, =0. 
These conditions, therefore, can be written as 
x) =0, x>0. (9.8) 


Inequality (9.7) shows that the evaluation of A, with respect to B, is 
negative; conditions (9.8) are equivalent to (9.5). The superscript (/) indi- 
cates that the parameters refer to the /-th basis B, in chain (9.6) (the para- 
meters associated with the basis B have no superscript). 

In the following we shall also require an alternative form of conditions 
(9.7), (9.8) in which the parameters of the successive basis B,,, appear. 

For the vector A, to be suitable for introduction into the r-th position 
of the basis upon transformation from B, toB,, , it is necessary and 
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sufficient that 


At) >0, (9.9) 
xe) 0, xh > 0. (9.10) 


Here s, is the index of the vector A, introduced into the r-th position of 
the basis B,. To prove this proposition we apply the recurrence formulas 
(9.1) and (9.2). 

Since x «1 and A®=0, we obtain 


i+ Py 
x) = (9.11) 
rk 
l 
xe =< (9.12) 
(141) Ay 


The equivalence of conditions (9.8) and (9.10) follows from equalities 
(9.11) and (9.12). The equivalence of (9.7) and (9.9) follows from equality 
(9,13). 

9-3. In dealing with the possibility of a chain (9.6) arising, we set out 
from the simplest case. We assume that in all the iterations constituting 
the chain new vectors are introduced in the same position in the basis. We 
shall first establish the following auxiliary proposition: 

Lemma 9.1. Let B,,, be obtainedfrom B, by introducing the restraint vec- 
tor A, into the r-th basis position. If the vector A, is suttable to be intro- 
duced into the r-th position of the basis B,,,, then it is also suitable to be 
introduced into the r-th position of the basis B,. 

Proof. By assumption, 


AY) <0,. cf =0, xf >0. (9.14) 
From recurrence formulas (9.1) and (9. 2), 
Aw 

AUT a A —— xf, (9.15) 
x 

git) _ af (9.16) 


From (9.16) for /=0,q and from the last two relationships in (9.14), we 


obtain 


on oe wi “5 > pe (x) > 0). 


Further, since Af) <0, x? >0, and x0 >0, we have 


al) 
At? = Alt? 4 Se ma mm tra < bgt? <0. 


The vector A, thus satisfies conditions (9.7) and (9.8). This completes the 
proof. 
Now, for any /,0</<?#, let 
B, = (Any Ags +00) Agays Ag) (9.17) 
where A), = A;,=A,, B,=B. 


Theorem 9.1. A cycle (9.6) comprising bases of the form (9.17) cannot 
arise. 
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Proof of this theorem follows immediately from Lemma 9.1. Indeed, 
let uS assume the contrary. Then the vector A,=A;, is suitable to be intro- 
duced into the r-th basis position when transforming from B,_, to B,;=B. 
Hence, from Lemma 9.1, the vector A,=A;, is suitable to be introduced into 
the r-th basis position when transforming from B,_, to B;_,. Applying 
Lemma 9.1 in this way ¢ times, we conclude that vector A, is Suitable to 
be included into the r-th position of the basis when transforming from B to 
B,. According to condition (9.7) this, in particular, indicates that the eva- 
luation A, of the vector A, with respect to the basis B=(A,, A,, ..., Ag_,, 4,) is 
negative. On the other hand A,=0, since the vector A, enters the basis B. 
This contradiction proves the theorem. 

The number of zero components of the support program is called the 
degree of degeneracy (or simply degeneracy) of this program. 
It follows from Theorem 9.1 that cycle (9.6) may arise whenthe degree of 
degeneracy of the support program corresponding to the bases in chain (9.6) 
is greater than unity. The same conclusion can be drawn from the results 
of Chapter 4, §6 (see Chapter 4, Exercise 8). 

9-4, Theorem 9.1 shows that cycle (9.6) can arise only if new vectors 
are introduced into at least two basis positions. We will consider the 
simplest of all possible cases, i.e., we assume that new vectors in chain 
(9.6) are introduced into exactly two basis positions. 

To be Specific, let these positions be 1 and 2. In this case the basis B, 
is completely defined by the two restraint vectors in the first two positions 
(the vectors occupying the other positions remain unchanged). Therefore, 
for any of the bases (9.6) we shall indicate only the vectors occupying the 
positions 1 and 2. 

Consider the chain 


B(A,, A,)— B, (Ay, A,) — B, (A, A,) +B, (A,, 4.) — B(A,, A,). (9.18) 


Let us write the conditions under which this chain is possible. Since A, is 
introduced into the 1st position of the basis when transforming from B to 
B,, we have from (9.7) and (9.8) 


¥%,>0, A<0O. (9.19) 


In the next iteration, the vector A, is introduced into the 2nd position of the 


basis. Hence 
x) >0, AM <0. 


Expressing x‘) and Af) in terms of the parameters of B from recurrence 
formulas (9.1)and (9.2) (k=3, r=1), we have 
1) Xea 
Xe = X15 > 0, 630) 
AO = A,—= Xi <0. 
Now take the fourth link in the chain. Here the vector A, is substituted 
for A, in the 2nd basis position, resulting in the initial basis B. We apply 
conditions (9.9) and (9.10). In this case B,=B,, B,,,=B,r=2, s,=4. Hence 


x, >0, A,>O0. (9.21) 


Finally, in the third link of chain (9.18) the vector A, occupying the lst 
position is replaced by A, Again applying (9.9) and (9.10), we have 


xf? >0, AP >0. 


241 


We now express these parameters of B, in terms of the corresponding 
parameters of B. The basis B, is obtained from B by introducing the vec- 
tor A, into the 2nd position. Hence, from the recurrence formulas (9.1) 
and (9.2) 


x19 =x,,—Fx,,>0, A} =A,— 3 x,,>0. (9.22) 


Thus, if chain (9.18) is possible, inequalities (9.19)—(9.22) are satisfied. 
Let us now investigate this system. Applying the second inequality in 
(9.21), conditions (9.19), and the second inequality in (9.20), we obtain 


¥,, <0. (9.23) 


The second inequality in (9.20) can, therefore, be rewritten in the equi- 
valent form 


A, <A, =# (9.24) 


“Xu 
Comparing (9.24) with the second inequality in (9.22), we obtain 
x3 x, 
A, a <A, a 
But according to the second inequality in (9.21), A,>0. Hence 
Yay — Fa, (9.25) 
Xxg X16 


Applying (9.23), we rewrite inequality (9.25) in the equivalent form 
x, <2, (9, 26) 


Inequality (9.26) contradicts the first inequality in (9.22). This proves 
the inconsistency of system (9.19)-—(9.22). Chain (9.18) thus cannot arise. 

9-5. Let us now increase the length of chain (9.18) by one. We obtain 
a new chain of bases: 


B(A,, A,)—>B, (A,, 4,)—+B,(A,, A,) —~B,(A,, A.) +B (A,, A.) ~B(A,, A,). (9.27) 
Let us write the conditions for transforming from B to B, and from B, to 
B,, applying (9.7) and (9.8). Following the procedure in 9-4, we obtain in- 
equalities (9.19) and (9.20). 
The conditions of transformation from B, to B, and from B, to B, can be 


obtained from (9.9) and (9.10) as in the case of the chain (9.18). Omitting 
the details, we write out the corresponding inequalities: 


x, >0, A, >0, (9.28) 


4 x 
18 


9.29 


It now remains to express the conditions for transformation from B, to B, 
in terms of the parameters of basis B. Applying (9.7) and (9.8) to the third 
step of chain (9.27), we obtain 

xP >0, A <0. 
Applying the recurrence formulas (9.3) and (9.4) for i=r=1, r,=2, j=5, k=3 
k,=4, we obtain 


(2) __ ¥2a X1s— *10 Xa 0 
cae X aq Xig— Xa X14 oo (9.30) 
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AY?) == A, ~2m =X Xe A _ <as <a 7 A & 0, (9.31) 


XeaXiz—XenXiq © Lag Mig — Nyy Vig 


Thus, cycle (9.27) may arise only if inequalities (9.19), (9.20) and (9.28)- 
(9.31) are consistent. 
We shall show that these inequalities are inconsistent. From the first 
inequalities in (9.19) and (9.20) we obtain 
¥aa% 1a — X29 %, > 0. (9.32) 
Analogously, from the first inequalities in (9.28) and (9.29), 
ug X1y — 455 X14 > 0. (9.33) 
We shall distinguish between three possibilities depending on the sign of 
A = X45 Xy— X15 Nyy. (9.34) 
1. Let A=0. Then, from (9.31), applying the second inequality in (9.19) 
and the inequalities (9.32) and (9.33), we obtain 
A, <0. 
On the other hand, from (9.28), 
A,>0. 


Hence, the first case is impossible. . 
2. Let now A>0O. In this case, inequality (9.31) can be transformed to 
the equivalent form 


Xue X19 — Xas % Xeq Xap —%as © 
A a Xas ar fa a ea =e 4,. (9 : 35) 


From (9.35) and the second inequality in (9.20), we obtain 


X10 Feu Xig—XgaX%1e 4 _ Fee %ig— 7 ns Kg 

A, X13 > A> Xa5 X19 %15 X29 A, Xas X19 * 10% 05 A,. (9 -36) 
Applying inequalities (9.36), (9.32), (9.33), and the second inequality in 
(9.28), we obtain (for A>0) 


x Xag Xyg— 245 X 
a ae Tr est) 


Since A, <0 (the second inequality in (9.19)), obvious transformations re- 
duce (9.37) to 

Xas (ye Xeg— Ft Xa) 

yy (Xa X19 — X15 Xan) ae: (9.38) 
Hence, applying the first inequalities in (9.19), (9.28), and condition (9.32), 
we see that A is negative. The second case is also impossible. 

3. It now remains to aralyze the third possibility, namely 4<0. In this 

case inequality (9.31) is equivalent to the condition 


Xaq X19 — Xa X X15 Xeq— X05 *% 
A, Sop Fu— Biya 1 Kaa Bug 8g Be Ay. ee) 


Since A, <0 (second inequality in (9.19)), A<0, and inequality (9.33) applies, 
we have 


X15 %gq— Xa *wA >0 
$3 ° 


Xa5 419 215 Xn 
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The second inequality in (9.29) and inequalities (9.39) thus lead to 


Xog X19 459 X14 A, > A, > it Ay. (9.40) 


Xa X1g— X15 X22 
From inequality (9.40) and second inequality in (9.28) it follows that 


tn (Fis uF Xn) Sg, (9.41) 


¥y5 (Xgq Xyg— X15 Xe) 


From (9.41), together with the first inequalities in (9.19), (9.28), and in- 
equality (9.33), we have 
A = Xyg Ky — XX gy > 0. 


Once again we obtain a contradiction. The third possibility is also ruled 
out. 

Thus we have proved the inconsistency of inequalities (9.19), (9.20), 
(9.28)-—(9.31) associated with the sequence of bases (9.27). This indicates 
that the cycle (9.27) cannot arise. The preceding analysis culminates in 
the following general proposition: 

Theorem 9.2. A cycle of length less than six iterations long cannot arise 
when solving by the simplex procedure. 

Proof, Consider an arbitrary cycle of a linear-programming problem, 


Let new vectors be introduced into { basis positions in the course of this 
cycle. The length of the cycle, obviously, cannot be less than 2. There- 
fore, if there exists a cycle whose length is at most 5, then t<s. Con- 
sequently, we should consider only those cycles in which new vectors are 
introduced into one or two basis positions only. According to Theorem 9.1 
no cycle in which new vectors are introduced into only one basis position 
can arise. Therefore, it remains to analyze the case €=2, The minimum 
length of a cycle with =2 cannot be less than four steps. Hence, it suf- 
fices to study cycles of length 4 and 5 with (=2. 

If in two successive iterations of some cycle the new vectors are intro- 
duced into the same basis position, the length of the cycle can be reduced 
by one. Indeed, according to Lemma 9.1, the vector introduced into the 
basis in the second iteration is Suitable to be introduced into the same 
basis position in the preceding iteration. Hence, the second iteration can 
be omitted, which reduces the cycle length by one. This shows that in the 
analysis of cycles of length 4 and 5 we may limit the discussion to chains 
of form (9.18) and (9.27), respectively. We have, however, proved that 
neither of these chains can arise. This completes the proof. 

9-6, According to Theorem 9.2, any attempt to construct a cycle of 
less than six iterations a priori cannot succeed. 

Consider a cycle of six bases of the form 

B(A,, A,)—+B, (A,, A,) +B, (A,, 4.) +B, (A,, Ay) — 
—B,(A,, A,)—~B,(A,, 4,) — B(A,, A,). (9.42) 
In parentheses we give as always the vectors occupying the first two posi- 
tions of the basis. The remaining basis positions remain unchanged (¢=2). 

We write the conditions necessary and sufficient for chain (9.42) to arise. 
First, the first two basis variables x,, and x,, of the support program 
X corresponding to cycle (9.42) should be zero. The first three iterations 
of chain (9.42) coincide with the first three iterations of chain (9.27). The 
conditions of transformation from B to B,, from B, to B,, and from B, to 
B, therefore have the form (9.19), (9.20), and (9.30)-(9.31), respectively. 
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The conditions of transformation from B, to B, from B, to B,, and from 
B, to B, are best obtained applying (9.9) and (9.10). Omitting the details, 
which do not differ from the corresponding transformations for chains (9.18) 
and (9.27), we give the final results: 


x%,>0, A,>0 (9,43) 
(transformation from B, to B); 


x\t = Kgs ay > 9, 


9.44 
A") a A, 3 x4, >9 : 
(transformation from B, to B,); 
xh) = Taunt > 0, (9.45) 
As es jo Pega Sd * A __ ¥14 X20 * eg X16 A, >0 (9.46) 


XisXag— rg Xop 9 X yg Xag— Xig Xe 


(transformation from B, to B,). 
To derive the last two formulas, we applied equalities (9.3) and (9.4) for 
i=r=2, r,=1, k=6, &, =5, f—4. 
The first inequalities in (9.20), (9.44) and inequalities (9.30) and (9.45) 
are obviously equivalent to the conditions 
Hyg My — X yy XQ > 0, 
Xiy Mag — Xy5 Xqg > 0, 
Mee % 13 — Kay Xi > 0. 


(9.47) 


The necessary and sufficient conditions for a cycle (9.42) to arise thus 
have the form 


I. x,, > 9, (9.48) 
x4, > 0, (9.49) 
X15 Xap — X35 %_ > 9, (9.50) 
X15%_,—%,, %,, > 9, (9.51) 
Xs6 Xi, — Xs, x4 0. (9.52) 
II. A, <0, (9.53) 
es (9.54) 
A,—3tA, <0, (9.55) 
A,—#A,>0, (9.56) 
Keg Xsp— Fa %as Q _ X05 X10 418 X02 
A, hie Xu—X 6 X33 A, Xie Xu X16 Xe3 A, < 0, (9.57) 
__ ¥e Xe— Xia Hag A X14 X20 X00 X10 9.58 

a. ¥15 Xs0— *10 *a5 A, X15 X20— *16 X25 a a ( ) 

%,_ = %,, = 0, 


In system (9.48)—(9.58) the first five inequalities guarantee the positive- 
ness of the corresponding coefficients in the expansion of the new vector in 
terms of the basis vectors. The last six inequalities in the system ensure 
negativeness of the evaluations of the new vectors with respect to the cur- 
rent basis. 

We shall prove consistency of system (9.48)—(9.58), isolating a certain 
class of its solutions. Let 


A, = A,= —A,= —A,=a>0, (9.59) 
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X,,=%,, =, 


o=%=B, (9.60) 
ee Y> 
Xs =x,,=6. 


Under these conditions, inequalities (9.53), (9.54) are automatically satis- 
fied. Inequalities (9.48), (9.49) are equivalent to the requirement 


B>0. (9.61) 

Inequalities (9.50)-(9.52) are equivalent to 
yt —a' > 0, (9.62) 
yi —ad > 0. (9.63) 


Applying (9.59), wecan easily show that conditions (9.55)—(9.58) are, respec- 
tively, equivalent to the inequalities 


a+6<0, (9.64) 
(y+ a) (B-+-a—y—8)>0. (9.65) 


System (9.48)—(9.58), subject to the additional assumptions (9.59), (9.60), 
is, thus, equivalent to system (9.61)-(9.65). Let us consider this system, 
comprising five inequalities. 

1. Let y>0. Then, from (9.62), 


(y-+a)> 0. (9.66) 
Applying (9.66) and (9.65), we obtain 
6<fp+a—y. (9.67) 


From (9.61) and (9.64), a<0. Hence, (9.63) can be rewritten in the form 


yp 
Comparing (9.67) and (9.68), we have 


are (9.69) 


On the other hand, from (9.64), ry <1, Therefore, 


Comparison of (9.69) and (9.70) gives the impossible relationship 
[Bta[< 0. 


Hence, if (a, B, y, 4) is a solution of system (9.61)-(9.65) y>0(for y=0 
inequality (9.62) is not satisfied). ; 

2. Let y<0. Then inequalities (9.63), (9.65) are, respectively, rewrit- 
ten in the form 


d>lle, a>Jyi—jpteal. 


ary 


Therefore, for 6>|y| both these inequalities are satisfied (according to 
(9.64), b< 1) The other three inequalities are equivalent to 


0<B<-—a<—y. 
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Therefore, any set (a, f, y, 5) satisfying the conditions 
0<Pp<—a<c—y<4, (9.71) 


is a solution of inequalities (9.61)—(9.65). 

Let us now choose arbitrary a, f, y, 6, satisfying (9.71). 

Applying formulas (9.60) we then determine the parameters x, Next we 
choose linear-form coefficients c, such that the evaluations of the corres- 
ponding restraint vectors satisfy inequalities (9.59). Then the parameters 
x;, and A,(i=1,2; j=3, 4, 5, 6) constitute a solution of inequalities (9.58)- 
(9.68) corresponding to cycle (9.42). We have thus obtained an entire class 
of linear-programming problems in each of which a cycle of six bases may 
arise. 

This result and Theorem 9.2 lead to the following proposition. 

Theorem 9.3. The minimum length of a cycle in linear- programming 
problems is six iterations. 


9-7, We now give an example illustrating the results of 9-6. 

Consider the following linear-programming problem with seven nonnegative variables having three 
equality restraints. 

Maximize the linear form 


L (X) =Xy,— Xa Xy— 


subject to the conditions 
x, + *%,—2x,—3x,+ 4x, =0, 


Xe t+4x,—3x,—2x,+ x, =0, 
Xyt Xt At %+4%,=1. 


The compilation of the initial tableau for this problem is obvious (Table 5,20). We see from this tableau 


TABLE 5.20 


ty | 
aaa! ae ee ee a ee ee ee 
leau 
| [4 | 1 |-2|—2 4 | 
lL tal Lede] | fe 
3 o 


that the problem in question belongs to the class of problems constructed in 9-6, Here 


=X y= X= — 2, 
=X4g=Xy= 1, 
=Xy=%,,.= —3, 
= Xqg 2 1, 

X19 2% 29 0, 
a=—A,=—A,=—A,=A,=1. 


Conditions (9,71) are obviously satisfied, Therefore, when solving this problem by the simplex method, we 
may encounter cycling, We give here a sequence of tableaus (see Table 5,21) corresponding to the cycle in 
question, 

In the first step the vector A,(A,=—1) is introduced into the basis, Here 6,=0 is obtained on the first 
two positions of the basis, The restraint vector A, is placed in the first vector position, Further, the vector 
A, (A= —1) is introduced into the basis. Since 6,==0 is obtained only on one basis position (the 2nd), 
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TABLE 5.21 (1-6) 
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the vector A, is placed in the 2nd position. In subsequent iterations, the vectors A,;, A,, A;, A, are intro- 
duced in the Ist, 2nd, Ist, 2nd basis positions, respectively. 

We see from Table 5,21 that the basis position in which a new vector was to be introduced in the lst, 3rd, 
and Sth iterations was not chosen in any specific manner, In these cases the position was chosen arbitrarily, 


It is obvious that had we used the exact rule described in 2-3 for choosing the position, no cycling would 
have arisen, 


For example, let us apply the exact rule in the first iteration: 


Following this rule, we compare the ratios 
Ti and xu _ 9 
+13 X23 
Hi Fn 


Since X, “a x,,' the vector A, should be introduced into the second basis position, The reader will verify 
that, having applied this rule, the linear form is increased already in the next iteration. 

9-8, The results of the present section show that an arbitrary choice of 
the position in which a new vector is introduced into the basis may produce 
cycling. We have, however, seen that for cycling to occur the problem 
parameters should satisfy fairly rigid requirements, As expected cycling 
is, therefore, a rare phenomenon. 

Indeed, in none of the linear-programming problems solved in the applied 
examples has a cycle arisen although most of them have been solved without 
using the exact rule for elimination of one of the basis vectors, The first 
published example of a cycle is due to Beale /6/. In the literature there are 
also some references to an unpublished example of a cycle constructed 
earlier by Hoffman /36/, 

We thus reach the following conconlusions. 

1. Although in principle cycling may occur, applied linear-programming 
problems should be solved with the aid of the simplified rule for choosing 
the vector to be eliminated from the basis. One of these rules is given in 
§ 2. 

2. In theoretical applications of the simplex method, to avoid cycling 
the exact rules developed in Chapter 4, §6, must be applied. Otherwise, 
the class of problems for which any results obtained will be valid is a priori 
limited. 


EXERCISES TO CHAPTER 5 
1, Consider a two-parameter (a and 6) family of linear-programming problems, 


Maximize the linear form 
G 
LiX)= 3 xy 
zl 


X,+2x,— x, +4x%,+6x,+ x,=2, 

X,— 3x, + 4x, + bx, -+ax,+6x,=7, 

x, +5x,—2x,+ x, +6x,+axr,=5, 
x,=20, j=l, 2,..., 6. 


subject to the conditions 


Indicate the domain of definition of the parameters @ and 6 corresponding to case (a) (optimal program), 
case (b) (unsolvable problem), and case (c) (improvable program) with reference to the following support 
program of the problem: 

X=(I, 2, 3, 0, 0, 0). 


Give a graphical representation of the corresponding domains in the (@, 6)-plane, 
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2, Applying the first simplex algorithm, maximize the linear form 


subject to the conditions L (X) = 4, + 2x,—%, + 44, — 3%, 

Xp Xgt3x,— x, +2x,< 10, 
2X5 +3x,-- 2,06, 

3x, + 4eg+ x, +2e,+ x, 25, 


x, 0, j=l,..., 5. 


3. Solve Exercise 2 applying the second simplex algorithm. 

4, Apply the product form of the second algorithm to the solution of Exercise 2, 

5. Solve Exercise 2 applying the comments of 5-3 on the determination of the vector to be introduced 
into the basis. 

6, Draw a block diagram of the product form of the second algorithm. 

7. Solve Exercise 2 with additional restraints 


x, <l, j=l, ) 5, 
considering the problem as a problem with bilateral restraints, 
8. Determine a support program of a linear-programming problem with the following system of restraints: 


3x, —2x,-+ x,—3x,+ 4x, =2, 
5x, +3x,—2e,-+ ut x,=6, 
X,— 2x, + 4x, — 2x, = 3, 
4,20, J=1,..., 5 
9, Determine a support program of a problem with the following system of restraints; 


SX, + 2x.— X,+3x,— X,+2x,=6, 

2x,—3x,+2x,+ x%+2x,+ x,—1, 

16x,— 5x, + 4x, +9x, +4x,+ 7x, = 16, 
x20, j=l, 2,..., 6 


Write the maximum number of linearly independent equations of the system and write the equivalent 
problem. 


Hint: Use the comments of 8-1, 
10. Maximize the linear form 


L (X) =X, + Xg— 3x q + y+ 5 


subject to the restraints of Exercise 8: 
(a) applying the support program of Exercise 8 as the initial program; 
(b) applying the M-method. 
11, Prove formulas (9.3) and (9.4). 
12, Consider the following linear-programming problem: 
Maximize the linear form 


L (X) =x,—*%4+%,—x, 
subject to the conditions 
X, +2x,—3x,—5x, + 6x, =0, 
X,+6x,—5x,—3x, + 2x, =0, 
3X, + X%+2x%,4+ 4x,+2,=1, 
x,20, j=1, 2, cees 7. 


Taking the vectors (A,, Ay, A,) as the initial basis, show that a cycle may arise if solution of the 
problem is attempted without applying the rule which guarantees agains cycling, 
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Chapter 6 


THE DUAL SIMPLEX METHOD 


Several methods suitable for solving linear-programming problems fol- 
low from the duality theory discussed in Chapter 3. From a synthesis of 
the simplex method and the principal concepts of duality we have the so- 
called dual simplex method, which was first described by Lemke 
/73/ in 1954, 

The solution of a linear-programming problem by the dual simplex method 
(which is sometimes simply called the dual method) involves determin- 
ing an optimal program of the dual problem and hence, by the duality 
theorems, an optimal program of the primal problem, too. 

In Chapter 3, 5-5, we showed that the components of the optimal pro- 
gram of the dual problem give an estimation of the influence of the various 
restraints of the linear-programming problem on the magnitude of the 
maximum of the linear form, The method presented in this chapter pro- 
ceeding from approximate (preliminary) evaluations of restraints of the 
primal problem (from an initial program of the dual problem), yields, after 
successive refinements of these evaluations, a vector of exact evaluations 
of the problem restraints (an optimal program of the dual problem). The 
system of preliminary restraint evaluations obtained in each iteration of 
the dual method can be made to correspond to an n-dimensional vector X 
which satisfies the equality restraints of the primal problem, but may have 
negative components. The system of restraint evaluations obtained in the 
last iteration corresponds to a vector X* with nonnegative components. The 
optimal support program is, thus, obtained as the result of successive re- 
finement of restraint evaluations; this is reflected in the name given to 
the method in Soviet literature—the method of successive eva- 
luation refinement. 

We note that when passing from a given iteration to the next the value of 
the linear form decreases monotonically. Thus, unlike the simplex method 
where the maximum of the linear form is approached from below, the dual 
method leads to the optimum from above. 

The dual simplex method is presented as follows. First, in §1, we state 
the optimality test and discuss the theoretical principles of the dual method 
with reference to the nondegenerate case. In §2 the formal description of 
the method is supplemented by two geometrical interpretations. §3 deals 
with applications of the dual simplex method to linear-programming prob- 
lems with bilaterally restrained variables. 

In the first three sections the dual problem is assumed to be nondege- 
nerate. In § 4 we establish rules which can be followed when applying the 
dual method to degenerate problems; these rules guarantee against cycling 
in the degenerate case. 


251 


In § 5 and § 6 we discuss two computational procedures, two algorithms 
of the dual simplex method. Each of these dual algorithms proceeds from 
a given initial support program of the dual problem. In §7 we consider 
various methods for determining the initial support program of the dual 
problem. In the last section, § 8, the simplex and the dual simplex methods 
are compared and the problem of applying them simultaneously is considered. 


§ 1. Principles 


1-1. We write the linear-programming problem in canonical form, 
Maximize the linear form 


L (X) = die, (1.1) 

subject to the conditions 
Pudi zky= By g=1, 2,..., m; (1.2) 
x, 20, P= 1, Qed M, (1.3) 


As always we assume the matrix ]a,,la,. of restraints (1.2) to be of rank 
m. The dual problem of problem (1.1)—(1.3) involves minimizing the linear 
form 


L(y= Loy, (1.4) 
subject to the conditions 
m 
PT ea J=1,2,..., 1. (1.5) 


The primal and the dual problems can be written more compactly in the fol- 
lowing form: 


L(X)=(C, X)—>max subject to the conditions AX™=B, X=0; 
L(Y)=(B, Y)>min subject to the conditions YA2S}C. 


Let us now define some concepts associated with the dual problem. 

A program Y of the dual problem is said to be a support program if 
among the restraints (1.5) reduced to equalities on this program there 
exist m linearly independent restraints (see Chapter 2, § 5). 

The basis of a support program Y of the dual problem (1.4)-(1.5) is 
any system of m linearly independent restraint vectors A, of the primal 
problem such that 


(Y,A,)=¢s, p=, Deis m, 
or, equivalently, 
m 
ZF iy Yi= Soy p=1, 2, eee, M, (1.6) 
Let J, denote the set of indices (s,,s,,..., s,) of the basis of the support 
program Y. The system of vectors A,,..., A,,, comprising the basis of some 


support program of the dual problem is called, for brevity, the dual 
basis. 
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A supportprogram Y of the dual problem is said to be nondegenerate 
if for any vector A, not appearing in the basis 


(Y, Aj)>¢, (1.7) 


(see Chapter 2, § 5). 

The dual problem (1.4)—(1.5) whose support programs are all nondege- 
nerate is calleda nondegenerate problem. Geometrically, nonde- 
generacy indicates that exactly m boundary hyperplanes intersect at each 
vertex of the polyhedron (polyhedral set). 

In the following sections the dual problem is assumed to be nondegenerate. 

Each support program Y of the dual problem (more precisely, each 
support-program basis) is conveniently made to correspond to some a-di- 
mensional vector X satisfying restraints (1.2) of the primal problem. This 
correspondence will enable us, in the following, to translate all the con- 
structions pertaining to support programs of the dual problem into terms 
of the primal problem. 

Let us expand the constraint vector B in terms of the dual basis (A,, ... 

., As) Let x, be the corresponding expansion coefficients, The a-di- 
mensional vector X whose s,-th components (i=1, 2, ..., m) are identical with 
x, and whose other components are all zero will be called a pseudopro- 
gram of the primal program. All x, will be called basis components 
of the pseudoprogram, and the others extrabasis variables. The vec- 
tors of the dual basis, like those of the primal basis, can be characterized 
not only by their indices but also by their respective positions in the basis. 

The basis component x, of the pseudoprogram will be denoted by x,, ac- 
cording to the position i of the vector A, in the dual basis. The pseudopro- 
gram components obviously satisfy restraints (1.2) of the primal problem: 


tuts= 2 QisyXsy = Op f= 1, 2, ..., m. (1.8) 


It must be kept in mind that some x,, may be negative, since X, generally 
speaking, is not a feasible program of the primal problem. In what follows 
the terms dual basis and pseudoprogram basis will be used synonymously. 

A pseudoprogram can also be defined independently of the dual problem. 


Let S=(A,, ..., As,) bea system of linearly independent restraint vectors. 
Let / be the system of indices s,, ..., s,. We expand the restraint vectors 
A, (=1, 2, ..., a)and the constraint vector B=A, in terms of the vectors of 
systems S: 
m 
A, = p> XyjAns (1 9) 
B=A,= & XigAs- (1.10) 
=1 
We take 
mm 
Ay= 2ituXy—ep jJ=1, 2, eee, a. (1.11) 


Theorem 6.1. An n-dimensional vector X such that x,=x, and x,=0 
for j¢I ts a pseudoprogram if and only if all A,>0 (j=1, 2,..., 2). 

Proof. The vectors A,,...,A,, are linearly independent. We may, there- 
fore, compute the vector Y=(y,, ...; y,) whose components satisfy the 
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conditions 
m 
> Cus, = Cs. 
w= 
We have 
m m m m m 
A, = z C5, X53 — F = p> (2 ausy,) X1j—- Gg = 2 Mu (2 Bys,X 1) —C;. 


It follows from (1.9) that 


au — p> Gus X pss (1 e 1 2 ) 
therefore 
Ay= 2eyty—y= Be GY, — Sj (1.13) 


This relationship shows the equivalence of the two definitions of a pseudo- 
program, This completes the proof. 

We must emphasize that while a feasible program of a problem is speci- 
fied by the problem restraints only, a pseudoprogram is defined by both 
the restraints and the linear form of the problem, 

The dual simplex method involves application of the simplex method to 
the dual problem. This approach gives a new technique for solving the 
primal problem. 

Consider a support program Y=(y,, ..., y,) of the dual problem. The 
basis of support program Y comprises the vectors A,, ..:, As, This cor- 
responds to pseudoprogram X with the basis components x,,(i=1, 2, ..., m). 

Optimality test. If all the basis components of pseudoprogram X are 
nonnegative, pseudoprogram X is an optimal program of the primal prob- 
lem, and support program Y is a Solution of the dual problem. 

Proof. We have the following chain of equalities: 


m mom m m m n 
L(Y)= p> biy i= pa (2 ais x, ) Ji= 2 oe i (2 Fis) P) = 2 €s,*s, =F p> c/%, = L (X). 


Here the second equality follows from (1.8) for pseudoprogram components. 
The third equality is obtained by reversing the order of summation. The 
fourth relationship follows from (1.6). The fifth equality holds because all 
the extrabasis components of a pseudoprogram are zero. 

With nonnegative x,, pseudoprogram X is a feasible program of the 
primal problem. According to Chapter 3, Lemma 1.2, the programs ofa 
dual pair related by the expression 


are optimal programs of the corresponding problems, This completes the 
proof, 
Observe that the above optimality test follows directly from the second 
form of the simplex optimality test, since the requirements of the second 
form of the test are satisfied for pseudoprogram X. In the general case, 
the above test is only sufficient to establish program optimality. We leave 
it to the reader to prove that this test is also necessary, if the program Y 
of the dual problem is nondegenerate (see Exercise 1). 

1-2. Let a support program Y of the dual problem with the basis 
As,, --+-, Ay, be given. This corresponds to pseudoprogram X of the primal 
problem, The components in the expansion of the constraint vector B=A, 
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and of the restraint vectors A, of the primal problem in terms of the vec- 
tors of the dual basis can be computed from the equalities 


Ay= Ya xiyAy j=0, 1, 2, eee, A, (1.14) 


The parameter x,, in (1.14) is the coefficient in the expansion of the restraint 
vector A, in terms of the vectors of the dual basis associated with the vec- 
tor A, occupying the i-th position in the dual basis. 

In what follows, we shall require an alternative technique for computing 
the coefficients x;,;: 


Xiy= 21 %jCi0 i==1,2,...,m; /=0,1,2,...,2, (1.15) 
where e;,, are the coefficients in the expansion of the m-dimensional unit 
vectors e, (w=1, 2, ..., m) in terms of the vectors of the dual basis, i.e., 

Il eis Wim =(Asy» Ags oer Asg)™’s (1.16) 


Formula (1,15) follows from the matrix relationships 


or 
X, = Ay'Ay, 
where 
X= (X14, Xi oe, Xj)" Ay=(As,; Ag, eee, A,,). 


We distinguish between three cases depending on the signs of the basis 
components of the pseudoprogram and of the coefficients in the expansion 
of the restraint vectors A, in terms of the dual basis: 

a) The basis components x, are nonnegative for all i(i=1, 2, ..., m). 

b) Some of the x,, are negative and for at least one of them all x,;,20 
for j=1,2,..., 2a. 

c) The pseudoprogram has negative basis components x;,, but each has 
some negative coefficients x,,(j=1, 2, ..., a). 

In case (a), according to the optimality test, the pseudoprogram is an 
optimal support program of the primal problem. In what follows we shall 
show that in case (b) the linear-programming problem is unsolvable (the 
restraints of the primal problem are inconsistent), and in case (c) we may 
advance to a new support program of the dual problem and, consequently, 
to a new pSeudoprogram with a lower value of the linear form, 

1-3. To simplify the analysis of cases (b) and (c), it is advisable to 
follow the variation of linear form (1.4) under transformation from support 
program Y to a new program 


Y (6) = Y+ be", (1.17) 
Here es (e,;,, ..., im)s €;, are the coefficients in the expansion of the unit 
vectors 
TN, 
e,=(0, ..., 0, 1, 0, ..., 0) 


in terms of the basis vectors of program Y. To simplify notations in 
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relationships containing the vector e”, the number of the basis to which the 
components of this vector correspond is not indicated. 

The transition from program Y to program Y(6) will be called the ele - 
mentary transformation of program Y associated with the vector A, 
of the dual basis. 

Inserting the components of the vector Y (6) = (y, (6), ..., y,, (6)} into the left- 
hand side of restraints (1.5) of the dual problem, we obtain 


2 ed () = z A Yt 6 p>) 77% ips 
From (1.15) 


In particular, for / € /,=(s,, 5, ...) Sa) 


< 0 for JE ly, J#S;, 
p> 9.8 =) 1 for j=s;. 


We have from these formulas 


2 Our JE ly, l#Sy 
2. Fp (8) = 2 Hue 8, J= Sy ay) 
p> aie +Ox;,, SE ly. 
We have previously defined in (1.11) 
A= 2 4% ep j= 1, 25 ees, A, 
Since Y=(y,, .... Y,) iS a program of the dual problem, we have 
A,20 for j=1, 2, ..., a. (1.19) 


The vector Y(#) is a feasible program of the dual problem if and only if 
m 
4, (6) = 2a 44,9, (8) —e, > 0, 9 a Rarer eee 2 (1,20) 


Let us now determine the conditions under which all (1.20) are satisfied. 
Applying (1.11), (1.18), and (1.20), we obtain 


A,=0, if / € ly, IFS y 
A , (6) = A,+6=6, if j= Sty (1.21) 
A, +x; if JE ly. 


It follows from (1.21) for j=s, that 620. If x,,20, then A,()>A,>0. If 
x;,<0, conditions (1.20) are satisfied if 7 


y(6) is thus a feasible program of the dual problem (1.4)—(1.5) for all 9 
in the interval 


0<(<6,, (1.22) 
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where 


A 
6.= min f\, 1.23 
. sia ( i) ( ) 
If x,,20 for j=1,2,...,”, § is bounded above, i.e., §,=oo. For a nondege- 


nerate program all A,>0, /#/,, so that 6,>0. 
The value of the linear form L(Y) of the dual problem on program Y(6) is 


L(y (6)] = 2 by, (0) = ») b, (y, +8¢;,). 


According to (1.15), for /=0 we have 


so that 
L[Y (@)]} =L(Y) + 6x,.. (1.24) 


1-4, We will now analyze cases (b) and (c) outlined in 1-2, 

Let case (b) obtain: among the negative components x, of pseudoprogram 
X there is a component x,, for which x,,>0, j=1, 2, ..., a. In this case Y(§)— 
the result of the elementary transformation (associated with the vector A,,) 
of program Y—is a feasible program of the dual problem for any §. Hence, 
we see from (1.24) (for i=r) that linear form L(Y) is unbounded below in the 
set of feasible programs of the dual problem. According to Lemma 1.3, 
Chapter 3, this is possible only if the restraints of the primal problem are 
inconsistent. 

Note that in the simplex method unsolvability of a problem resulted from 
unboundedness of the linear form in the set of feasible programs. Thecase 
of inconsistent restraints did not occur, since in advance we assumed the 
existence of an initial program of the primal problem. When analyzing the 
dual simplex method, we start with an initial program of the dual problem. 
This ensures boundedness of the linear form of the primal problem, but 
does not guarantee consistency of its restraints. 

Consider case (c). Let the pseudoprogram have negative basis compo- 
nents such that each has some negative coefficients x; In this case the 
vector Y (6) is a feasible program of the dual problem for any §>0 not ex- 
ceeding 6, in (1.23). Take a negative basis component x,,<0 of pseudopro- 
gram X. By assumption, some of the x,, are negative. Let the index j 
on which §, is attained be k: 


Obviously, 
A, (§,) = A,+4,+,, = 0. 


According to Chapter 4, Theorem 2.1, the system of vectors obteined 
from the dual basis of program Y when the vector A, is replaced by A, is 
linearly independent, since by assumption x,,#0 (x,,<0). Hence Y’=Y(6,) is 
not just a feasible program, but is actually a support program of the dual 
problem. The basis of the support program Y’ is obtained from A,, ..., Asp 
when As, is substituted for A,. 

By assumption, the problem in question is nondegenerate, so that Y’ is 
a nondegenerate program, i.e., A,(§,)>0 for j¢/y.. Hence, 0, is obtained on 
a single vector A,. In the nondegenerate case the vector to be introduced 
into the basis is thus uniquely determined. 
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We observe from (1.24) that when passing from program Y to Y’=Y(6,), 
the linear form of the dual problem decreases by 


18,x;)[>0 (>0, %;.<0). 


Observe that elementary transformation of support program Y into sup- 
port program Y’=Y(6,) corresponds to a transition from pseudoprogram X 
to pseudoprogram X’ of the primal problem. 

The value of the linear form L(X) of the primal problem on the pseudo- 
program is equal to the value of the linear form L(Y) on the corresponding 
support program of the dual problem, 

Indeed, 


n m m nie 
L(X)= »» (= 2 c/x)= 2 Cy Xio= 2 Xing 2 aus. = 


= 2% p> Gusti 21 Yb, = L(Y). (1.25) 


Thus, in case (c) elementary transformation of the support program Y 
of the dual problem associated with the vector A, decreases the linear form 
L(Y). In terms of the primal problem this means that in case (c) we may 
pass from pseudoprogram X to a pseudoprogram xX’ with a lower value of 
the linear form L(X). 

Successive iransformations from a given support program Y of the dual 
problem to another (or, in terms of the primal problem, successive trans- 
formations of one pseudoprogram to another) are continued until a solution 
of the problem is obtained or unsolvability is established. 

Each transformation from a given pseudoprogram to the successive one 
constitutes an iteration (step) of the dual simplex method. When 
this method is applied to the solution of problems with a nondegenerate dual 
problem, we may not return to a basis which has once been examined. 
Otherwise, the support programs of the dual problem obtained in different 
iterations would be coincident. This, however, is impossible since the 
linear form of the dual problem decreases monotonically. The number of 
iterations required to reach a solution of a nondegenerate problem (or to 
establish its unsolvability) is a priori bounded by the total number of bases 
of the dual problem, which does not exceed Cy. 

1-5. We now briefly outline the sequence of operations involved ina 
Single iteration of the dual method. Each iteration comprises two stages. 
In the first stage we check whether the pseudoprogram is a feasible pro- 
gram of the primal problem and, if not, whether the problem is solvable. 
To this end we compute the basis components of the pseudoprogram (ex- 
pand the constraint vector in terms of the vectors of the dual basis) and 
establish their signs. 

The pseudoprogram proves to be a feasible program and, consequently, 
a solution of the problem (case (a)) ifall its components are nonnegative. If 
the optimality test is not satisfied, we must verify that the restraints of 
the primal problem are consistent. 

Case (b) (unboundedness of the linear form of the dual problem below 
and, consequently, inconsistency of restraints of the primal problem) arises 
if for some negative basis variable of the pseudoprogram x,,<0 all the co- 
efficients x,, are nonnegative. 

Finally, case (c) is encountered if for any i, such that x,<0, some of 
the x,, are negative. 
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The first stage of the iteration thus terminates inone of the three possi- 
bilities (case (a), (b), or(c)). If case(a)or(b) holds, the solution process is 
terminated. In case (c), we proceed with the second stage of the iteration. 
The second stage entails determining the elementary transformation which 
will produce a new support program of the dual problem and, corres- 
pondingly, a new pseudoprogram of the primal problem with a lower value 
of the linear form. In the second stage we choose the position r of the dual 
basis with a negative pseudoprogram component and determine the vector 
A, to be substituted for the vector A, in the initial dual basis. The new dual 
basis is, thus, obtained from the old basis when the restraint vector A, is 
replaced by A,. In the second stage, moreover, we compute all the para- 
meters required for determining and investigating a new pseudoprogram. 
The current pseudoprogram is analyzed in the first stage of the successive 
iteration. 


§ 2. Geometrical interpretation 


2-1. Following the two geometrical interpretations of the linear-pro- 
gramming problem we shall consider two geometrical interpretations of 
the dual method. 

We start with the first geometrical interpretation. The polyhedral re- 
straint set M of problem (1.1)—(1.3) is contained in the intersection of 
hyperplanes defined by equalities (1.2). The support programs of the prob- 
lem correspond to vertices of the polyhedral set. Each vertex is formed by 
the intersection of 2 independent hyperplanes (m hyperplanes correspond 
to restraints (1.2) and a—m hyperplanes correspond to zero components of 
the support program). 

We give the geometrical interpretation of a pseudoprogram, The com- 
ponents of pseudoprogram X satisfy restraints (1.2) of the problem, all the 
extrabasis components x, (/¢/,) being equal to zero. Moreover, the dual 
basis defining the pseudoprogram corresponds to nonnegative values of the 
parameters 


™m 


A= p> Cy,%;,—C, 20 


for j=1, 2, ..., a. Hence, in the first geometrical interpretation, a pseudo- 
program corresponds to a point X at which a independent hyperplanes inter- 
sect (m hyperplanes defined by restraints (1.2) and n—m corresponding to 
the extrabasis components x,=0 of the pseudoprogram), In the general case, 
however, the point X corresponding to pseudoprogram X is located outside 
the polyhedral set M. To refinethe geometrical definition of the pseudo- 
program, we must establish the geometrical meaning of the conditions 


A,=0, J=1, 2, ooe, A. 
We define a polyhedral cone Kx by 
D 4, ,= by i=l, a; eee, mM, (2.1) 
{=1 
X20, fly =(Syy Sqr oe ey Sm) 
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The cone kK, is spanned by some of the hyperplanes of the polyhedral set 
M passing through the point X. Hence, the cone Ky, has its apex at the 
point X. 

We impose another condition on the geometrical analog of the pseudo- 
program which amounts to requiring that the corresponding polyhedral cone 
Ky, lie in the lower halfspace defined by the linear-form hyperplane passing 
through the point X. In other words, the vector C pointing in the direction 
of increasing linear form and the cone Ky should lie on either side of the 
linear-form hyperplane passing through the point X. We shall now show 
that this requirement corresponds to inequalities 


A ,=0, JN, 2) Sing 


The last requirement together with 


defines the pseudoprogram. 

Thus, let S={A,, ..., A, } be a linearly independent system of restraint 
vectors, and x, and x,, the coefficients in the expansion of the constraint 
vector B and the restraint vectors A, in terms of the vectors of system S. 
Let, moreover, the s,-th component of the z-dimensional vector X coincide 


with x, (/=1, ..., m), and all the other components of X be zero. Then, for 
any n-dimensional vector X’, 
L(X')=L(X)— J} Aj, x), (2.2) 
lé¢ly 
where /J,=(s,, ..., S,). Relationship (2.2) is verified in the same way as 


formula (4.6) in Chapter 5, § 4. 
Now, let the point X be a pseudoprogram of the problem, and let x’ 
belong to cone Ky. Then 
A, 20, Jey 2s cing th 


(X is a pseudoprogram), and 
x20, jel,  (X' EKy). 
We see from (2.2) that in this case 
L(X') <L(X), (2.3) 


i.e., any point of the cone K, and the vector C lies on either side of the 
linear-form hyperplane passing through the point X. 

Conversely, let it be known that the cone Ky, is in the lower halfspace, 
i.e., any point of the cone satisfies inequality (2.3). We take a point 


X’=(x', ..., ¥,) of the cone kK, such that 
x, —Ox;, for jis. 4= 1, 2}. 22am, 
x= 1) for pl, p#/, (2.4) 
6>0 for p=/. 


Conditions (2.4) for 0<6< oo define the rays spanning cone Ky. It is easy to 
verify that for the point X’ 


L(X')=L(X)—6A,. (2.5) 
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Comparing (2.3) and (2.5), we obtain 
A, =>0. 


We have thus established a correspondence between the geometrical 
image of the pseudoprogram and its analytical definition. 

The point X, i.e., the geometrical analog of a pseudoprogram, will be 
called a pseudovertex of the polyhedral restraint set M. 

It must be emphasized that a pseudovertex, like a pseudoprogram, is 
defined not only by the polyhedral restraint set, but also by the linear form 
of the problem, Moreover, we note that some of the hyperplanes consti- 
tuting a pseudoprogram need not be boundary hyperplanes of the polyhedral 
set M. This refers to the hyperplanes corresponding to inequality restraints 
which follow from the other restraints. 

2-2. Let us now geometrically interpret, in terms of the primal prob- 
lem, the result of elementary transformation of the support program Y of 
the dual problem into Y’=Y¥Y(6,). In other words, we give a geometrical 
interpretation of the transformation of pseudoprogram X into pseudoprogram 
xX’ = X(6,). 

We take the hyperplane x, =0 corresponding to the negative basis com- 
ponent x, of pseudoprogram X. 

Hyperplane x, =0 may have some points in common with the polyhedral 
cone Ky, though not necessarily. If the intersection of the cone and the 
hyperplane is an empty set, the problem is unsolvable, its restraints being 
inconsistent. Indeed, in this case the entire cone kK, is on the same side 
of hyperplane x, as the pseudovertex X, for which x, <0. On the other 
hand, the cone K, contains the polyhedral restraint set. Besides (2.1) the 
coordinates of the points in the domain of definition of the linear form of 
problem (1.1)—(1.3) also satisfy the condition x,>0 for j€/, and, in parti- 
cular, x, 20. This contradiction will not occur only if the polyhedral re- 
straint set is empty. 

Now let the hyperplane x, =0 intersect the cone Ky. The intersection of 
the halfspace x, <0 and of the polyhedral cone Ky is the convex polyhedral 
set M,. The pseudovertex X is one of its vertices. Each of the newly 
formed vertices is adjoining to the point X. Now let the linear-form hyper- 
plane passing through the pseudovertex X undergo parallel translation until 
it touches the nearest vertex X’ of the polyhedral set M,. The first vertex 
of My reached by the linear-form hyperplane is a pseudovertex of the poly- 
hedral set M, i.e., a geometrical image of the pseudoprogram correspond- 
ing to the support program Y’. The program Y’=Y(9,) is obtained from the 
support program Y of the dual problem under elementary transformation 
associated with the vector A, , 

We now prove the validity of the preceding geometrical interpretation of 
the transformation of pseudoprogram X into pseudoprogram ’. 

Consider an arbitrary vertex of the polyhedral set My. This vertex is 
the intersection of m hyperplanes corresponding to equality restraints (1.2) 
of the problem with the hyperplane x, =0 and n—m—1 hyperplanes x,=0(j/€/,) 
corresponding to the extrabasis variables of pseudoprogram X. Since there 
is a total of n—m extrabasis variables, My, has at most ra—m vertices adjoin- 
ing xX. Let the vertex of M, for which x,>0(f€/,) be denoted by X;. To obtain 
the components of X, from the components of X, we must, following formu- 
las (2.4) for j=#, increase 0 until x) vanishes. To different vertices x, 
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of the set M, correspond the values § given by 
x, =x, —Ox,,=0. 


But x, <0, §>0. This indicates that the vertex X, exists only if x,,.<0. Hence, 
the values 6 corresponding to the vertices X, are computed from the condi- 

tion 

Xs, 


§=x,= 
Xrt 


for x,,<0. 

The parameter x,A, coincides, up to a factor of normalization, with the 
distance of the point X, from the linear-form hyperplane passing through 
the point X (see Exercise 2). Thus, closest to the linear-form hyperplane 
is the vertex X, such that 


- x 
x,A,= min x,A,= min — A). 
XA<o Xet<o *rt 


The basis component x, of pseudoprogram X is negative. Therefore, 


x,A,=|2,,|min(—t) . (2.6) 


X pt <0 Xrt 


Therefore, the index k on which the ratio —2 with x,,<0 attains its 
r 


minimum defines the vertex X =X, of the polyhedral set My into which 
pseudoprogram X is transformed. Following these considerations we 
choose the index & of the vector to be introduced into the dual basis under 
elementary transformation of support program Y into Y(6,). The bases of 
the support programs Y and Y’=/Y(§,) correspond to pseudoprograms X and 
xX’, respectively. 

We have thus established that pseudoprogram X’ corresponding to sup- 
port program Y’ is represented in this geometrical interpretation as the 
vertex X’ of the polyhedral set My closest to X. The point X’, like X, is 
a pseudovertex of the polyhedral restraint set M. 

2-3. Let us now describe, in geometrical terms, a single iteration. 

In the first stage of the iteration the pseudoprogram is tested for opti- 
mality. If pseudovertex X belongs to the polyhedral set M, the linear- 
form hyperplane passing through X proves to be the support hyperplane of 
the polyhedral set M at the point X, and the pseudoprogram X, being a 
feasible program of the problem, defines its solution (case (a)). 

Now let program X not be a feasible program of the problem. Then, 
among its basis components we find x, <0. For the hyperplane x, =0 to 
intersect the cone K,, it must intersect at least one of the rays (2.4) span- 
ning the cone. In other words, for some # the conditions 


i= Xs _— 6x,, =0 


should be satisfied. But x, <0, @>0. Therefore, x{ =0 is impossible if all 
Xx, are negative. In this case, the hyperplane x,,=0 does not intersect the 
cone Ky. We have already seen that if the hyperplane x,,=0 and the cone ky, 
are disjoint, the linear-programming problem is unsolvable, its restraints 
being inconsistent (case (b)). 

If the hyperplane x,.=0 intersects Ky, we proceed with the second stage 


of the iteration. The transformation from pseudoprogram X to pseudo- 
program X’ with a lower value of the linear form L(X) corresponds to dis- 
placement from point X to the nearest vertex X’ of the polyhedral set mM, 
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(case c). When passing from point’X to X’ the variable x, vanishes and 
the variable x, (see (2.6)) is made a basis component of the pseudoprogram. 
This parallel translation of the linear-form hyperplane decreases the 
value of the linear form in each successive iteration. Since the number of 
pseudoprograms is finite, the maximum of the linear form is obtained after 
a finite number of iterations. It is obvious from geometrical considerations 
that unsolvability of the problem (lack of feasible programs) will also be 
detected after a finite number of steps. 


2-4, We shall now illustrate the preceding considerations using Example 1 from Chapter 4, § 3, 
Example 1, Maximize the linear form 


L (X) =5x,—x,— 2x, + 5x, + 5x,— x, 
subject to the conditions 
—2x,+5x,+%, = 10, 
Xi %, +%, =1, 
x, +2x, +x, =6, 
10x, —3x, +x,=15, 
x, = 0, j=], 2, eee, 6. 


Solution, The corresponding dual problem is stated as follows: 
Minimize the linear form 


E (¥) =10y, + y+ 6y, + 15y, 


subject to the conditions 


—2y,+ 42+ ¥,+10y, > 5, (2.7) 
Si Yn + 2y,— 3y, > —I, (2.8) 
sy >—2, (2, 9) 

Ws >5, (2. 10) 

Ys > 5, (2.11) 

wel. (2, 12) 


We easily observe that replacing inequalities (2.7), (2.8), (2.11), (2.12) by equalities we obtain a system 
of four independent equations whose solution reduce restraints (2,9) and (2,10) to strict inequalities, Hence, 
the initial dual basis comprises the vectors A,, A,, A, and A,*. 

Let us now compute the basis components of the initial pseudoprogram and the coefficients in the expan- 
sion of the restraint vectors in terms of the vectors of the initial dua] basis, 

The basis components of the pseudoprogram satisfy the equations 


a 
B = > xioAs,, 
f=1 


where As, =A,, As, =Ay, As; =A, and As, =A,. Thus, xj) (é/ =1, 2, 3, 4) are obtained from the system of 
equations 

10 == — 2x49 + 5Xq0, 

1 =X y9— *g, 

6 = X49 + 2X29 +X qo, 

15 = 10 ¥, 9>— 3X a9 + Xo. 


Hence x,)=5, X= 4, Xy=—7, %=— 23 and, correspondingly, 

X =(5, 4, 0, 0, —7, —23). 
The linear form on pseudoprogram X is L(X)=9. Some of the pseudoprogram components are negative. 
Therefore, X is not a feasible program and as such does not define a solution of the problem, 


Let us expand the restraint vectors in terms of the basis vectors of pseudoprogram X, 
We. have 


a 
s=1 


* Methods for determining an initial support program of the dual problem are discussed in § 7, 
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Solving the corresponding systems of equations, we obtain the matrix 


1 

10 a 3 99, 

1 2 
Ot > > 0 0, 

ll xigll= 3 3 
00 — —3 10, 

44 
00-7 —Fo1 


Some of the coefficients x,, and x,, corresponding to the negative basis components of the pseudopro- 
gram are negative. There is, therefore, no reason to consider the problem unsolvable, 

We have thus obtained case (c) from the dual basis and this enables us to pass to a successive pseudo- 
program lowering the value of the linear form, The vector As,—A, corresponding to the negative basis 
component of the pseudoprogram with highest absolute value is eliminated from the dual basis. The vector 
A, to be introduced into the basis corresponds to the minimum 6, of the ratio —%L for negative x,;. The 


a] 
parameters A, are computed from (1,13): 


m 
Ay 2 eeity—* 
t= 


In our case 


C5,=G=5, Cy, =Q=—1. C,=c=5, ¢,=c=—l. 


Therefore 


T 
% 

ll 
wt 


d=5- 5H 4 +5 (-4H(—) (—F 
A=5- 3 +(— 5 +5(—-9+(—1) (— 


We compute @,: 


2 7 

_ 4, A,\ _ 3 3 7 

0, min ( les —Ft)=min 7’ i = 
3.63 


@, is thus obtained on A,. The vector A, should be introduced into the dual basis in place of A,=Ag,. 
The vector A, is introduced into the fourth position and is accordingly denoted by A,,. 

To compute the new pseudoprogram and the coefficients in the expansion of the restraint vectors in 
terms of the vectors of the new dual basis, we should solve systems of equations analogous to the preceding, 

We have 


eee Se a2 
“\ 44° 92° ~°44° 44° 


3 5 

10 qq °° ia’ 

5 l 

Jul=[ Bo 
*al 23 9 
00—7G O1-|- 

00 a l0—a. 
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The value of the linear form on pseudoprogram X’ is L (X’) = 4q° The signs of the component 24 


of the pseudoprogram and of the coefficients x,, indicate that we again have case (c). The vector As=A, 


with a negative pseudoprogram component should be eliminated from the basis, It is replaced by the vector 
A, on which 


13 7 
. A; 44 44 13 
0, = =e) Vi es eee 
° min ( =) min 3° 9 53 
44 44 


is obtained, All the coefficients Xi¢ in the expansion of the constraint vector B in terms of the vectors 
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of the new basis are positive, We have thus obtained the optimal program 


, (48 45 101 20 
x = (5. 3° 3° 93° % 0), 
nm 93 
L (xX = 55° 


2-5, We shall now use this example to give the geometrical interpretation of the solution of a linear- 
programming problem by the dual method, 


As in Chapter 4, 4-2, we construct the problem equivalent to Example 1 which will enable us to inter- 
pret the principal stages of the iterations in the plane (x,, x,). 


-2 J Fi2é 4 6° 
\ 
X 
Ae ‘\\ 
™ SW 
\ 
FIGURE 6.1 


The equivalent problem is stated as follows: 
Maximize the linear form 
L (X)=%,+%, 
subject to the conditions 
—2x,+5x, <— 10, 
4, ; a ~< 1, 
x, +2x,< 6, 
10x, — 3x, <= 15, 
x,>0, x, >0. 


The equivalent problem (and consequently Example 1) corresponds to the domain of definition of the 
linear form shown in Figure 6.1. The sides of the polyhedron are defined by segments of the straight lines 
x,=0, x,=0, 
%, = 10+ 2x,—5x, =0, 
x%,= t— X,+ x,=0, 
X,= 6— x,—2x,=0, 
x, = 15— 10x, 4+ 3x, =0, 
Striation along the line indicates the halfplane in which the corresponding variable is nonnegative, The 
arrow OG defines the direction in which the linear-form line (hyperplane) is to be translated to increase L (X). 
The initial pseudoprogram X=(5, 4, 0, 0, —7, -—-23) can easily be shown to cortespond to the point 
G (5,4) at which the lines *,=0 and x,=0 intersect. The point G is a pseudovertex of the polyhedral 
restraint vector M (polyhedron OABDEF). Indeed, the polyhedral cone Ky with its apex at the point X 
(in our case, the plane angle AGF) lies in the lower halfplane with respect to the linear-form line (hyper- 
plane) M’N’ passing through the point G. 
We will now illustrate, geometrically, the transformation to a current pseudoprogram. The line 
%=20(xs,=0) separates the pseudovertex G and the restraint polyhedron M. At point G@ the component 


X, =X, =— 23 <0. 
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The line x,=0 meets the cone Ky. The intersection of the halfplane x,<,0 and the cone Ky isa 
triangle GG@’E (polyhedral set My). The first vertex of the triangle reached when the line M’N’ is translated 
: 105 65\ . 
parallel to itself in the direction of decreasing linear form is G’, The point G’= (Fr 55) is a pseudo- 


vertex of the polyhedron M, i.e. , a geometrical image of the pseudoprogram 


105 65 69 101! 
’ — —— or — sees 
a =(3  35°9 ag a ,0) 
Analogous constructions enable us to pass from pseudovertex G’ to point D(G") corresponding to pseudo- 
program X”. Pseudovertex D is also a vertex of the restraint polyhedron, Pseudoprogram X” is therefore a 
program, and consequently a solution, of the primal problem, 


2-6, We now give an example terminating in case (b)—the linear-programming problem is unsolvable. 
Example 2, Maximize the linear form 


L (X) =5x, + 4%, +4; 
subject to the conditions 


xX, +2x%,—x, = 2, 
4x, +3x, +, = 12, 
4x, +4x,+%,+%4+%,=10, 


x, 20, j=l, eno, 5 


Solution. In the dual problem we minimize the linear form 


L (VY) = 2y, +12y,+10y, 
subject to the conditions 


y,+4y, +44, > 5, (2, 13) 
24, +34, + 4y, => 4, (2. 14) 
—hW + y,>0, (2.15) 
Yt ys=0, (2. 16) 
y,2>1. (2.17) 


It can easily be seen that if in (2.13), (2.14), and (2.17) the inequalities are replaced by equalities, the 
solution of the system of three linearly independent equations will reduce restraints (2,15) and (2,16) to strict 
inequalities, Hence, the initial dual basis can be formed from the restraint vectors A,, A,, and Ag. 

To obtain the corresponding pseudoprogram of the initial problem and the relevant parameters, we solve 
the system of equations 
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Here Ay m Ay, As,= Ay, As,=Asy. We have 


x=($ ee 0, 0, —3) 


5° 5 5 

10 2 + 0, 
layl=||01 —2—50, 
oo 2 ty 


The basis components Xp and Xg» of the pseudoprogram X are negative, and the expansion coefficients 
X,, of all the restraint vectors in the dual basis, associated with the vector A,, are nonnegative. 

We thus have case (b). The problem is unsolvable, its restraints being inconsistent, 

To obtain a geometrical interpretation of case (b), we replace, as in Example }, the given problem by 
the equivalent problem from which the negative variables x5, x,, %, have been éliminated and the equality 
restraints replaced by inequalities. 

The equivalent problem is stated as follows: 
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Maximize the linear form 
L (X) = 4x,4+%, 
subject to the conditions 
X,-f 2x, >> 2, 
4x, + 3x, & 12, 
X,+3x,< 0, 
4,20, 42> 0 


The sides of the restraint polyhedron M defining the domain of definition of the linear form are defined 


by the lines (Figure 6, 2) 
x,=0, x,=0, 
X,=—2+ x, +2x,=0, 
X= 12—4x,—3x,=0, 
= — X,—3x, =0. 


The striation along the lines and the arrow OC serve the same purpose as in the preceding example. 


FIGURE 6.2 
Iti Bi uk ee 18 4 6 
t is easy to verify directly that the initial pseudoprogram X= °F: 0, 0, ma corresponds to 
the point of intersection F - -5) of the lines x,a:0 and ¥,=0. The cone Ky in our case coincides 


with the plane angle AFD. The cone Ky lies in the lower halfplane with respect to the linear-form line 
M’‘N’ drawn through the point F. Thepoint F is, thus, a pseudovertex, 


The line x,=s0 corresponding to the negative component of the pseudoprogram (Xs= x, == — . 


does not intersect the cone Ky (the plane angle AFD). This is possible, as we have seen, only if the re- 
straint polyhedron (the domain of definition of the linear form) is an empty set. 

The above argument is given in order to illustrate case (b) in the dual simplex procedure and give its 
geometrical interpretation, In fact, however, unsolvability of Example 2, i.e. , inconsistency of the prob- 
len) restraints, is immediately obvious from the restraints of the equivalent problem, and also from Figure 6, 2, 


2-7. We now give a geometrical interpretation of the dual method cor- 
responding to the second geometrical interpretation of the linear-program- 
ming problem. 

In Chapter 3, 1-2, we described the geometry of a pair of dual prob- 
lems in the (m+ 1)-dimensional space of points U=(u,, ..., U,4,). We remind 
the reader that the augmented restraint vectors 4,//=1, 2, ..., ”) span, in 
the (m+ 1)-dimensional U-space, a convex polyhedral cone kK corresponding 
to the set of n-dimensional vectors with nonnegative components. Moreover, 
in the U-space, we consider the line Q such that 

u,=b,, be N, Ze oe 
Uns, =F, — co<t<oo. 


To feasible programs X of the primal problem correspond those and 
only those points U which belong to the cone K and to the line Q. To the 
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optimal program of the problem corresponds the highest (in the sense of 
the Ou,,,-axis) point of intersection of the line Q and the cone K. 

The geometrical image of the set of feasible programs of the dual prob- 
lem in the (m+ 1)-dimensional U-space coincides with the set of all hyper- 
planes passing through the origin and extending above the cone K. (The 
cone K and the vector (0, ..., 0, 1) lie on opposite sides of the hyperplanes. ) 
The optimal program of the dual problem corresponds to the hyperplane 
passing through the origin and the highest point of intersection of the line Q 
and the cone K and extending above the cone K. 

Let us now clarify the geometrical meaning of the dual method. The 
augmented basis vectors of the initial support program of the dual problem 
define a hyperplane II passing through the origin and extending above the 
cone K. The hyperplane II contains m augmented vectors A, and, hence, 
contains some m-dimensional face of the cone K._ If the vector correspond- 
ing to the highest point of intersection of the line Q and the cone K belongs 
to this face, the hyperplane II specifies the optimal program of the dual 
problem (case (a)). 

Now let the line Q meet the hyperplane [II at point X above the highest 
point of intersection of the line Q and the cone K. The point X corresponds 
to some pseudoprogram of the primal problem. Let some basis component 
x,, of the pseudoprogram be negative, and all the expansion coefficients x,, 
of the restraint vectors in the dual basis associated with the vector A, be 
nonnegative. This indicates that in the halfspace u,,,—0 we may construct 
a (m—1)-dimensional hyperplane H,_, Separating the point B (specified by 
the constraint vector) and all the points A, (specified by the restraint vec- 
tors), The hyperplane H,,_, of the m-dimensional space z,,,=0 is spanned 
by the basis vectors A,,A,,...,A,,_.5A A The equation of the hyper- 


Spam?“ Sr4i? °° Sme« 


plane H,_, can be written in the form (A, U)=0, where U isa point in the 
halfspace u,,,=0, and the vector A is orthogonal to the vectors 4,, ..., A,_,, 
Ay Seg Age e., (A, A,,)=0 for i=1, ..., r—l,r+1, ..., m. To be specific, 


let (A, A,,)=1. Computing the scalar products (A, A,) and (A, B) we have 
(A, A)= Pa %7(A, Ag,) = %,,(A, A,,)=*,p 


(A, B)= > Xin (A, A,,) =~,,- 


But x,,<0, x,,>0. Hence, points A, and the point B are on opposite sides 
of the hyperplane (A, U)= 

In the (m+ 1)- ei U-space consider the hyperplane H, spanned by 
the vectors A Ag,_y» As,44) «++» As, and the vector e,,,. The hyperplane H, 
separates the line Q and the cone K spanned by the augmented vectors A,. 
This follows from the fact that H,_, separates the constraint vector and the 
restraint vectors of the problem in the subspace uw,,,=0. Hence, in our 
case, the line Q and the cone K are disjoint. The primal problem has no 
feasible programs (case (b)). 

Now let the line Q meet the cone, Let As, be the augmented restraint 
vector corresponding to a negative component of the pseudoprogram. We 
now rotate the hyperplane II about the vectors A,,...,As,_,, Ag ca ears Ape. 0 
that the point of intersection of the hyperplane and the line Q descends (in 
the sense of the Ou,,,-axis). The rotation is continued until the hyperplane 
If captures one of the augmented vectors A,, e.g., A,. The vectors As. 


5,’ eeeg 
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., Aya» Ans Agpyys e+es Ag COnStitute the next dual basis. The hyperplane 

II, defined by these vectors corresponds to the new support program of the 
dual problem. The pointofintersection of hyperplane II, and the line Q is 
the image of the current pseudoprogram xX’, The value of the linear form 
L(X) on pseudoprogram X” is less than on pseudoprogram X (the point X’ is 
below the point X). If after testing the pseudoprogram xX’ for optimality 
and the problem for unsolvability we do not obtain cases (a) or (b), we 
apply to hyperplane II, a procedure analogous to that previously applied to 
hyperplane If. Successive rotations of hyperplanes, corresponding to suc- 
cessive support programs of the dual problem, eventually produce a hyper- 
plane II* meeting the line Q at the point X*, which is a boundary point of the 
cone K. The hyperplane corresponds to the optimal support program of 
the dual problem, and the pseudoprogram X* is a program and, consequently, 
a solution of the primal problem. 

In the dual simplex method we thus approach the optimal program of the 
primal problem (the highest point of intersection of the line Q and the cone 
K) not from within the cone, as in the simplex method, but from without. 

2-8, We shall now illustrate geometrically the application of the dua] method to the determination of 
an optimal program of the problem solved in Chapter 4, 4-3, by the simplex method. 

The problem is: 

Naximize the linear form 

L (X) = 10x, + 8x, +7x, + 16x, + 21x, 
subject to the conditions 


4x, + 2x,+ 5x,110%,+ 5x,= 6, 
Ox, + 10x, + 12.5x, + 18x, + 16.5%, = 14, 


x, = 0, j=1, 2, oe 


In Figure 4, 3, Chapter 4, the vectors A, and the constraint vector B, the corresponding augmented 
vectors, and the line Q passing through the point B parallel to the Ou,-axis are shown. 


“3 


3 


FIGURE 6,3 
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The plane [1 defined by the augmented vectors A, and A, passes through the origin and extends above 
the cone K spanned by the augmented restraint vectors. Hence, the plane II is the image of a support pro- 
gram of the dual problem, and the point of intersection B, of the plane II and the line Q corresponds to 
pseudoprogram X with the basis (Ay, Ay,). 

In Figure 6,3 the plane II spanned py the augmented restraint vectors A, and A, is shown, The two- 
dimensional cones (plane angies) spanned by the augmented vectors A, and A, and the restraint vectors A, 
and A, are striated. 

The point B, is outside the striated cone (angle), i.e, , the component &, of pseudoprogram X is nega- 
tive. The vector A, should be eliminated from the basis. We rotate the plane II about the vector A, so 
that the point of intersection of the plane and the line Q descends, The first vector to be captured in this 
rotation is the vector A,. The plane II, formed by the vectors A, and A, is the image of the successive 
support program of the problem (Figure 6, 4). 


us 


eT Ay ~ 
ee Ci 


Oma 
A 


ame, 
L°L4 
42, ’ 


FIGURE 6.4 


The plane II, meets the line Q at the point B’ which is a boundary point of the cone ‘K. The point B’ 
is, thus, the image of the optimal program of the problem, The basis components of the support solution 
coincide with the coefficients in the expansion of the constraint vector B in terms of the restraint vectors A, 


The optimal program of the problem is, thus, 


x! =(F 0, 0, 0 i): 


and A,. We have Lot and Le=a- 
21" "21 


The maximum possible value of the linear form is 


L (X) =B'B= = ‘ 


The components g,, y, Of the optimal program Y of the dual problem coincide with the first m==2 com- 
ponents of the direction vector of the plane II, (the vector Y=(y,, Ya. «+++ Ya» Yass) Orthogonal to TI, and 
normalized by the requirement ya4,=—1). 


The equation of the plane JI, is 
Yi4,+Y lg = Us. 
The points A, (44, Gg, ¢,) and A, (aj, Ggg, 6) belong to the plane II,. Therefore 


YiOyy HY gFq, = Cy,  Yyg + ggg = Cs, 
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From these equations we compute the components y, and y, of the solution of the dual problem, In 
our case 


A,=(4; 9; 10), A,=(5; 16.5; 21), 


so that the optimal program of the dual problem coincides with the vector v=(—3 , ai): 


§ 3. The case of bilateral restraints 


3-1. Consider a linear-programming problem with bilateral restraints: 
Maximize the linear form 


L(Xy)=D ce, (3.1) 
f= 
subject to the conditions 
2 Apt /=B, (3.2) 
Here A,=(d,;, Gj: ++» Ge)’ and B=(b,, 4,, ..., 5,)". As always, the rank of 
matrix (A,, A,, ..., 4,) is taken as equal to m. 


Problem (3.1)—(3.3) was discussed in Chapter 4, § 5, where it was solved 
by the simplex method. Now, as then, we shall not assume all the variables 
to be bilaterally restrained. Some of them may have only one boundary 
value, In this case, the other boundary value (a, or f,) is assumed infinite 
(a,=—oco or f,= 00). 

In this section we describe the dual simplex method as applied to prob- 
lems with bilateral restraints. 

3-2. Let us state the dual problem with respect to problem (3.1)—(3.3). 
According to the general principles of duality (see Chapter 3, 4-1), this 
problem entails minimization of the linear form 


Ly, 2’, 2) = BoB, a,z;+ 2), Bz) (3.4) 

subject to the conditions 
2 219) 63 z+; pm Cy, Jl, 2, eos, A; (3,5) 
z0, fer; 20, Jer. (3.6) 


Here / (f) is the set of indices j for which a,>— co (B,;<oo), 
a= {0 it jel, 


140, if jer’; 
& — 1, if JE P, 
/ 0, if jer. 


It is left to the reader to verify that problem (3.4)—(3.6) is indeed dual with 
respect to problem (3.1)—(3.3) (see Exercise 3). 
Let the vector (Y; 2’; 2"), where 


Y=(y, Ky en Ia Zl her, Z= {hen 
be a feasible program of the dual problem. Let 


A= 2 aii ep jJ=1, 2, eeey n. (3.7') 
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Then 


A,=0, if JEL (B, = 00, a,;>— 00), 


A,<0, if ss (a= —00, B<co). (3.7) 


(4p Ney 


Indeed, for /é/, we have from (3.5) 
A, = 8) 2, —8) 2=6) 2,0. 
The second requirement of system (3.7) is verified in the same way. 
Now let Y=(y,, 4) +.) Yx) be an m-dimensional vector satisfying conditions 


(3.7). Then, there exist z, jeE/’, and z, jE/’, such that the vector (Y, 2’, 2’) 
is a feasible program of problem (3.4)—(3.6). Indeed, it suffices to take 


? A, if A,=>0, : 
=| 0, if A,<0 Vé!'), (3.8) 


~_{ —A, if A,<0, 
0, if A,>0 VES). (3.9) 


If je, JEP, then, from (3.7), 4,<0, and correspondingly 
(Ay, Y)—8,2,4+6) 2,=(A, Y)+2,=(Ap Y)—A,=e,. 


The case /€/’, jé” is verified in the same way. For /él’, jer (the variable 
x,of the primal problem has two finite boundary values) 


(A,, y)— 6, z, +8, 25=(Ap Y)—z, +2). 


From the definition of z, and z Uél, JEL) 


z,—z)=A,. 


Hence, the vector (Y, Z’, 2’) satisfies the j-th equality in (3.5). Observe that 
the case /€/', j¢/ is impossible, because each variable of the primal prob- 
lem is assumed bounded at least on one side. By definition, 2,20, z= 0. 
The vector (Y, 2’, 2") is indeed a feasible program of the dual problem, 

With fixed Y satisfying (3.7), z, for /¢” and z for sgl’ are uniquely de- 
fined: 

z=; Zi= —A,. 

Now if je/’, jE’, the j-th equality in (3.5) is satisfied, as we have already 


seen, if Se 
£,—Z,=A,, (3.10) 


Hence, if there exist bilaterally restrained variables x,, to each vector Y 
satisfying conditions (3.7) there correspond several feasible programs of the 
dual problem. 

From these programs we choose that which is associated with the least 
value of the linear form (3.4). For any /€/’, /”, we find negative z and Zz) 
satisfying equality (3.10) and such that —a,z,+ 6,2’ is conditionally minimized. 
It is easily observed that these z, and z, are determined from (3.8) and (3.9): 


“=| A, if A,>0, 


0, if A,<0; 
2” = —A,, if A,<0, 
/ 0, if A,>0. 


Thus, the 2, j€/',2’,/€/", which, together with the fixed vector Y, specify 
a program of problem (3.4)—(3.6) minimizing L(Y, 2’, Z”) are computed from 
(3.8) and (3.9). 

Inserting these z, and z) into the linear form of the dual problem, we 
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fe 


m n 
L(Y)=L(Y, 2’, Z")= dy bi — Bu v/Ap (3.11) 
=1 
where 
ay, if A,=0, 
Ys B, if A,<0. 

Since we are concerned with programs of the dual problem correspond- 
ing to the least possible value of the linear form (3.4), we shall consider 
only those of the programs whose components z, and z) are uniquely deter- 
mined by (3.8), (3.9). Remembering this, we may consider instead of the 
dual-problem programs the vectors 


Y=(Yi) Yer oor Yaa) 


satisfying conditions (3.7). 

3-3. We introduce some definitions, 

A vector Y=(y,, yy ws Y,) Satisfying conditions (3.7) is called the pro- 
gram kernel of problem (3.4)—(3.6). 

Applying the definition of a support program given in Chapter 2, §5, we 
may verify that the vector Y is a kernel of a support program of prob- 
lem (3.4)—(3.6) if and only if conditions (3.7) are satisfied and among the 
vectors A, j=1, 2, ..., a, there exist m linearly independent vectors A,,, 

A , 4s. such that 


$3? eee m 


As,=(As,, Y)— es, =0, i=1, 2, ..., m (3.12) 


(see Exercise 4). 

The system of linearly independent restraint vectors Ay,, /=1, 2, .... m for 
which (3.12) applies is called the basis of kernel Y of a support program 
of the dual problem. 

The set of indices of the vectors A, constituting the basis of kernel Y will 
be denoted by 4. In our case /,=(5,, S,,...,5,). 

It is easy to show that the support program (Y, 2’, 2”)of problem (3.4)— 
(3.6) is nondegenerate if and only if its kernel satisfies the condition 


(see Exercise 5). 

In this section the dual problem (3.4)—(3.6) is assumed to be nondege- 
nerate, The kernels of all the support programs of the problem should, 
therefore, satisfy condition (3.13). 

As in the case of the linear-programming problem in canonical form, it 
is advisable to make each support program of the dual problem correspond 
to some n-dimensional vector X satisfying equations (3.2). 

Let Y= (y,, Ys, «-,¥,) be the kernel of a support program of problem (3.4)— 
(3.6). We define the vector X=(x,, x,, ..., %,) as follows: 


(a) if fly, 


_ fa, for A,>0, 
x=} B, for A,<0; (3.14) 
(b) the other components of the vector X are computed from 
x,A,=B— 2, x,A, 3.15 
Pa me Po Ay (3.15) 


The vector X obviously satisfies restraints (3.2) of the primal problem. 
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The vector X associated with the kernel Y of a support program of the 
dual problem (or, more precisely, with the basis of this kernel) by rela- 
tionships (3.14) and (3.15) will be called a pseudoprogram of the 
primal problem (3.1)—(3.3). 

Let X be the pseudoprogram corresponding to kernel ¥. The system of 
vectors A, j/€/,, constituting the basis of kernel Y is sometimes called the 
basis of pseudoprogram X. 

The components x, of pseudoprogram X are called basis components 
for jél,, the remaining being extrabasis variables. 

In the nondegenerate case, to each kernel of a support program there 
corresponds a unique basis and, consequently, a unique pseudoprogram, In 
the degenerate case come kernels may have several bases and, consequently, 
correspond to several pseudoprograms. The application of the dual method 
to a problem with bilateral restraints involves passing from one support- 
program kernel of the dual problem to another (or, equivalently, from one 
pseudoprogram of the primal problem to another) monotonically decreasing 
linear form (3.4). 

3-4. The principal feature of the dual method, as of any other linear- 
programming method, is the test for finding the solution of the problem, 

Optimality test. If pseudoprogram X is a feasible program of the primal 
problem, i.e., if its basis components satisfy restraints (3.3), it solves 
this problem. The corresponding vector Y is then the kernel of the optimal 
program of the dual problem. 

Proof of the first part of this proposition follows directly from the second 
form of the optimality test established in Chapter 4, §5, since according 
to (3.14) and (3.15) program X fulfills all the requirements of this test. 

We shall require only the first part of the optimality test. As regards the 
second part of the test referring to the kernel Y, its proof is left to the 
reader (see Exercise 6). 

Consider the kernel Y=(y,, y,, .... ¥,) Of some support program of the dual 
problem. 

Let the restraint vectors A,, A,, ..., As, constitute the basis of this kernel, 
so that /,=(S,, S,, ....8,). Applying formulas (3.14) and (3.15) we associate 
pseudoprogram X=(x,, x,,...,%,) with the kernel Y and its basis A, jeé/,. The 
system of vectors A,/j€/,, cunstitutes the basis of pseudoprogram X. 

We now expand in terms the basis vectors of pseudoprogram X all the 
restraint vectors A, and the vector 


A,=B— XAp 
r >) A; 
We have 
Ay= 2xiAus /=0, 1, oe ft, (3.16) 


As always, x,, denotes the expansion coefficient of the vector A, asso- 
ciated with the basis vector A, occupying the i-th position in the given basis. 
In particular, for/=0, the expansion coefficients in (3.16) coincide with 
the basis components of the pseudoprogram, i.e., x,=x,, i=1, 2,...,m. 
According to (3.7') 


A= 3 ayy— ep 
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where Y=(y,, ¥..--».¥n) is the support-program kernel of the dual problem. 
By assumption, A,=0 for j€/y. 

In § 1 we showed (see (1.13)) that the parameters 4, can also be com- 
puted from the x,;: 


j= Qyou kis Gj (3.17) 
=3 


Given the parameters x,,; and A, computed from the given kernel Y (or, 
equivalently, from the corresponding pseudoprogram X), we may establish 
optimality of X (if it is indeed optimal) or construct a new pseudoprogram 
X’ corresponding to a kernel Y’ which is associated with a lower value of 
linear form (3.4) than Y. These parameters also enable us to establish 
unsolvability of the primal problem. Asin the canonical case, there too 
three mutually exclusive cases depending on the relative signs of the para- 
meters x,, and A; arise: 

(a) Ail the basis components of pseudoprogram X satisfy restraint (3.3), 
i.e., 

O,<%,,= x, <f, for i=1, 2,...,m. (3.18) 


(b) Some of the basis components do not satisfy conditions (3.18). Among 
these there is a component x,,=.x, such that 
0 ter A'So 
=9 for A,>0, : 
*y{ <0 for A <0 U€/y) 712) 
or 
0 ior Ao 
<0 for A,>0, 
ty =0 for A,<0 Ul). eo) 

(c) Among the basis components of pseudoprogram X there are some 
which do not satisfy restraints (3.18). However, for x,<a,, (3.19) do not 
hold, and for x,,>§,, (3.20) do not apply. 

If conditions (3.18) are satisfied(case(a)), pseudoprogram X meets the 
requirements of the optimality test. Hence, the vector X solves the primal 
problem, 

To analyze the two other cases, we find it expedient to use a concept 
analogous to elementary transformation of a dual-problem program dis- 
cussed in § 1. 

3-5. As in §1, |le;,||, denotes the matrix constructed of the coefficients 
in the expansion of the unit vectors e,, e,,...,e, in terms of the basis vec- 
tors of kernel Y (the basis of pseudoprogram X). Thus, 


lege Ane AgvsasiAg) 


To each restraint vector As;s l<i<m, we associate a transformation of 
the support-program kernel Y of the dual problem defined by 


Y (6) =(y, (6), ¥, (8), «++ 1 Ym (0)) = ¥+0e%, (3.21) 


where e=(e,,, €;,.++:€im) is the m-dimensional vector comprising the ele- 
ments of the /-th row of matrix |le,,||,. This transformation is the ele- 
mentary transformation associated with the vector A,. 

We now investigate the relevant properties of the vector Y(6) for different 
6. To this end we consider 


A, = 3 an Oey 
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From (3.21) 


A; O= % aynt+OR ayea—ey= A, +6 Y Cn. (3.22) 


Since the vector X,=(%,y) Xp... ,%,)? is a solution Bt dhe eyeter: Ghlinear 
equations 
(As, Ans oes > Ary) X= A, 


(see (3.16)), we obtain 


X= (Any Agrees 1 Asg) "Ay =ll ea Ile Ay 
Hence 
xy >») @ 7a, (3.23) 


i= 1, 2,...,m; j=0, 1, 2,...,a. 
Applying (3.22) and (3.23), we have 
In the following, we shall have to distinguish between two cases 
i)@>0, ii)6<0. 


Consider (i), Let 6 increase from 0 on upwards as long as all A,(6) 
maintain their respective signs. In other words, 6 should satisfy the con- 
ditions 
=0, if A,>0, 


b= A,+0x{ <0, if A ,<0. 


(From the assumption of nondegeneracy A,*#0 for j¢/,.) A departure from 
the first of these conditions is possible only if x,,<0, and from the second 
only if x,,>0. The two conditions are, therefore, equivalent to the inequality 


= ee : 
<6, = min ( i), (3.25) 
where the minimum is taken over all j for which at<0, x;,%9. If for any 


iti,, A, and x;, (x,,;#0) have the same sign, §,=00. Observe that since the 
dual-problem program with the kernel Y is assumed nondegenerate, 6,>0. 
Thus, in the first case 
0<6<6,, 


where 6, is defined by (3.25) and 6,=oo if the set of indices j on which the 
minimum in (3.25) is takenas empty. 

Consider (ii). Here 6 should be nonpositive. We again determine the 
limit of § for which the signs of A,(6) and A, coincide for all /€/,. In the 
same way as above we obtain 


A 
—O<0,= mine, (3.26) 


A 
where the minimum is taken over those /é/, for which 720 %,,;%0. When 
t 


none of the indices / have this property, 4, is taken equal to oo. 
Thus, for (ii) 
0<—O<6,, 
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where 6, is determined following the preceding rules. In the nondegenerate 
case 6,>0. 

The elementary transformation associated with the vector A, is applied 
with 6>0 only if a, >—oo and with §<0 only if B;, <oo. 

It follows from (3.24) that 


0,A#i (A=1, 2,...,m), 
bn O=4 4 =H. (3.27) 


The vector Y(6) for 6 satisfying condition (3.25) (@=0) or condition (3.26) 
(<0) therefore meets the requirements of inequalities (3.7). For /é/, this 
follows from (3.27) and from the preceding assumptions on the boundary 
values of the variable x,. For j¢/, this proposition follows from the defini- 
tion of 6, and from the validity of relationship (3.7) for kernel Y. 

The vector Y(6) is, thus, the kernel of some program of the dual problem. 

We now compute the value of the linear form (3.4) corresponding to the 
kernel Y (6). 


Applying formula (3.11), we have 


L@=LYOM=YAnO—Y v04,0, 


where 
a, for A,(6)=0, 


w= { B, for A,()<0. 


Since A,(#)A,;>0 for j€/,, As, (6)=As, = 0 for 4%i, and A;,()=9 we have 


= Vp if j#s,, 
vit) a,,(B.,), if j=s, and §0>0(6<0). (3.28) 


Here the y, refer to the kernel Y. 
We transform the expression for L(6) applying (3.21), (3.23), (3.24), and 
(3.28): at e 
L ()= Dy An+8 > b, en — » vA, (6)—8ys, ae 
A=1 A=1 lely 


= Son- FB vA +8 (BOF vardenl—valh 


Using our notations, 


bn— BY Q@, = by— Bi Xj% = Ayo) 
a ji ia 2 0 


therefore 


LO) = ZO BWA FLY teen — vad L (0) +8 (ti Yad. 
Thus, 
L(Y (6)) =L (Y)+9(x;,—Ys,)- (3.29) 
We recall that according to (3.28) 
_ f 4%, if 850 (case (i)), 
Ya \ B,, if 0<0 (case (ii)). 


3-6, We will now investigate cases (b) and (c) given in 3-4. 
i. We consider the two possibilities terminating in case (b). 
For some i=r, x,,<us, and x,; satisfy conditions (3.19). We apply to 


kernel Y the elementary transformation associated with the vector A, and 
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6>0 (case (i)). Since a,,>x,,>—oo, this transformation produces the kernel 
Y(6) for any 6, O<0<6,. Restraints (3.19) indicate that for any /¢/, either 


x,,=00r AL > 0. Hence, §,=oo. Y(6) is thus the kernel of some feasible pro- 
tf 


gram of the dual probelm for any nonnegative 8. 
Applying formula (3.29) (for i=r) and since y=, we have 


L(Y (6)) =L (¥)+8(x,,.—4as,). 
By assumption, *,,—a;,<0. Hence, 
lim L (Y (0)) = —oo, 


i.e., the linear form of the dual problem is unbounded below in the set of 
feasible programs. According to Chapter 3, Lemma 1.3, this implies that 
the restraints of the primal problem are inconsistent. 

The second possibility terminating in case (b), when for some i=r «x,,> 
and conditions (3.20) are satisfied, is analyzed analogously. Here we apply 
the elementary transformation associated with the vector As, and §<0._ It 
can easily be seen that in this case §,=oo and, consequently, the restraints 
of the primal problem are inconsistent. 

Case (b) thus amounts to unsolvability of problem (3.1)—(3.3), as the 
result of inconsistent restraints (3.2) and (3.3). 

2. Now consider case (c). We take the index r (a position in the basis 
of kernel Y such that 


i) x,,<as, 
or = 
ii) Xo >Bs,. 
Consider (i). We apply to kernel Y the elementary transformation as- 
sociated with the vector A, for §>0. Since case (b) does not obtain, 0,<oo. 


Let Y’=Y(0,), where 6, is determined from (3.25) with / substituted for r. 
Let 6, defined in (3.25) be obtained for j=, i.e., 


6, =—=__ Xrk . (3.30) 
Then 
A, (§,) =0. 
From the definition of elementary transformation it follows that for 
A=1, 2,...,7—1, 7+1,...,m, As, (6,) =A, =0. Hence 
m 
A, (6,) = p> a;,y,(8,) —c,=0 (3.31) 
LOT? J 2 8 i Gog aces (Spgs: Ry Spas ees pS 
Consider the system of restraint vectors 
As, As, , ete Asis A,, As,ary eee » As, (3.32) 
Since x,,#0 and the system of vectors A,,,...,As_,, As Asa. +++» As, CONSti- 


tuting the basis of pseudoprogram X is linearly independent, we have, from 
Chapter 4, Theorem 2.1, that system (3.32) is also linearly independent. 
Hence, applying inequalities (3.31), we find that Y’ is the kernel of a sup- 
port program of the dual problem. The basis of kernel Y’ is obtained from 
the basis of kernel Y when the vector A, is replaced by the vector A,, where 
k is determined from (3.30). 
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Since the problem is assumed to be nondegenerate 
A,(6.)#0 for Jély. 


Hence, equality (3.30) is possible for a unique j=k only. Therefore, in 
the nondegenerate case, condition (3.30) uniquely defines the index & of the 
vector to be introduced into the basis. 

According to (3.29), 


L (¥’)=L(¥) +6, (x,,—4s,). (3.33) 
As we have already indicated, in the nondegenerate case 6,.>0. Hence, 
L(¥')<L(Y). 


Thus, for (i) of case (c), we see that the elementary transformation asso- 
ciated with the vector A,,(x,,<a;,) and §>0 gives the kernel Y’ of a support 
program of the dual problem decreasing the linear form (3.4). 

Consider the second possibility (ii) when x,,>;,. In this case we apply 
the elementary transformation associated with the vector A, for 6<0. 
Proceeding as previously, we show that the vector 


y’ = Y(—6,) 


is the kernel of a support program of the dual problem. The basis of the 
kernel Y’ is obtained from the system A, /€/, when the vector A, is substi- 
tuted for A,. In this case the index ek of the vector A, to be introduced into 
the basis is determined from the condition 


An 6, (3.34) 


where 6, is computed from (3.26). 
The change in the linear form of the dual problem under transformation 
from Y to Y’ is determined from the equality 


L(Y’) =L(Y)—4, (x,,—Bs), (3.35) 
whence, applying the inequalities 6,>0 and x,,—B;,>0, we obtain 
L(Y')<L(Y). 


If the conditions of case (c) are fulfilled, we may proceed from the given 
kernel of a support program of the dual problem to the kernel of another 
support program decreasing the linear form (3.4). 

Let x’ be the pseudoprogram of the primal problem associated with the 
kernel Y’ and the basis Jy. Let us consider transformation from pseudo- 
program X to pseudoprogram xX’. The basis of pseudoprogram X (A, /€/y) 
is obtained from the basis of pseudoprogram X’ (A, j€ /y') when the vector A, 
is substituted for A,. The index & of the vector to be introduced into the 
basis is determined in (i) from (3.30) andin (ii) from (3.34). The extra- 
basis components x, of the pseudoprogram 4X’ for js, coincide with the 
corresponding components of pseudoprogram X, and 


n= { as, (in (i)), 


B., (in (ii)). 
The basis components of X’ can be determined from (3.15) with /y sub- 
stituted for /,. Observe that 
nr 


E(Y=L (X= Ms exp (3.36) 
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where X is a pseudoprogram of the primal problem associated with kernel 
Y of a support program of problem (3.4)—(3.6). The proof of equality (3.36) 
is left to the reader (see Exercise 7). . 

3-7. The dual simplex method when applied to a problem with bilateral 
restraints involves examining pseudoprograms of the primal problem, 

The method consists of identical iterations. Each iteration is subdivided 
into two stages. In the first stage the conditions specifying cases (a) and 
(b) are checked. If one of these cases obtains, the process of solution is 
terminated. If case (c) obtains, we proceed with the second stage of the 
iteration, in which a new pseudoprogram decreasing linear form (3.1) is 
constructed. 

Successive iterations are continued until a solution is obtained (case (a)) 
or unsolvability is established (case (b)). 

In the nondegenerate case each iteration decreases linear form (3.1), or 
equivalently, by virtue of (3.36), it decreases linear form (3.4). There is, 
therefore, no possibility of returning to a pseudoprogram which has pre- 
viously been examined, Further, since the number of various pseudopro- 
grams of the primal problem is finite, we say that the method is finite. In 
the next section we shall establish this point for the degenerate case. 

It follows from the preceding description of the dual method, that bila- 
teral restraints have almost no effect on the laboriousness of computations 
in a Single iteration, This will become even more obvious once we have 
considered the dual-simplex algorithms, discussed in 5-5 and 6-5. 

Worth mentioning is an additional advantage of the dual method connected 
with the determination of the initial pseudoprogram of the primal problem 
(or the kernel of a support program of the dual problem). Let all the 
variables x, of the primal problem be bilaterally restrainted by finite 
boundary values, Then /’=/"=(1,2,...,a). Conditions (3.7), which are ne- 
cessary and sufficient for the vector Y=(y,,y,,...,y,) to be the kernel of some 
feasible program of the dual problem, will therefore no longer restrict the 
choice of Y. 

Under these assumptions, any m-dimensional vector Y=(y,,y,,...,¥,) is 
the kernel of a feasible program of problem (3.4)—(3.6). Correspondingly, 
in this case, any linearly independent system of m restraint vectors can 
be adopted as the basis of the kernel of a support program of the dual prob- 
lem or of a pseudoprogram of the primal problem. 

Thus, if all the variables have finite bilateral restraints, the choice of 
the initial pseudoprogram is reduced to the choice of an arbitrary linearly 
independent system comprising m restraint vectors A, 

While in the simplex method bilateral restraints complicated the pro- 
cess by which the first approximation was determined, in the dual simplex 
method bilateral restraints, conversely, essentially simplify the determi- 
nation of initial data for the solution process. 

We remark further that when determining the elementary transformation, 
we took §, to correspond to the first zero of A,(#) for jé/,, This rule in- 
duces motion over adjoining vertices of the polyhedral restraint set of the 
dual problem, It may, however, turn out that when @ is further increased 
the vector Y(§) remains the kernel of a program of problem (3.4)—(3.6) and 
linear form (3.4) keeps decreasing. It is then convenient to exceed the 
limit 6, but the computations in a single iteration become in this case some- 
what more involved, However, the number of iterations is now reduced 
since several vertices at a time are crossed on the polyhedral restraint 
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set of the dual problem. This approach is closely connected with the so- 
called piecewise-linear problems /130, 31/. 

3-8. In conclusion we shall briefly review the sequence of operations 
constituting a single iteration of the dual method. 

Before starting with.a current iteration, we have a pseudoprogram X 
with the basis (A,,,...,A;,) and the corresponding parameters x,, and A, The 
first stage of the iteration starts with testing pseudoprogram X for opti- 
mality. If all the basis components of X satisfy restraints (3.3), we have 
case (a) implying that X is an optimal program of the primal problem. 

When some of the basis components exceed the corresponding boundary 
values, the parameters x,, and A, should be used to check the conditions 
corresponding to case (b) (see formulas (3.19) and (3.20)). If these condi- 
tions are satisfied, we conclude that the problem is unsolvable. Otherwise, 
we proceed with the second stage of the iteration. 

Among the basis vectors of pseudoprogram X corresponding to the basis 
components which exceed their boundary values we choose an arbitrary 
vector A,. ‘This vector is to be eliminated from the basis. The choice of 
the vector A, to be substituted for A, is made following different rules de- 
pending on the boundary that the basis component x,, has actually exceeded. 
If x,,<as,, the index k is defined by (3.30), and 6, is computed from (3.25). 
If x,,>fs,, the required index is obtained from (3.34), where §, is defined 
by (3.26). 

The new pseudoprogram xX’ and the corresponding parameters es and 
A, can be computed from previous data by recurrence formulas (see 5-5). 

Two different numerical techniques of the dual method are considered 
in 5-5 and 6-5. 


§ 4, Degeneracy 


4-1, Until now we presented the dual simplex method assuming nondegeneracy of the dual problem. In 
the present section we shall not make this restriction, First let us mention the difficulties which may arise 
when the dual method is applied to the degenerate case. For the sake of simplicity we shall limit the dis- 
cussion to a linear-programming problem in canonical] form (1.1)-(1.3). 

Let Y=(y,, Yg---.¥Ym) be an arbitrary support program of the dual problem (1.4)~(1.5) with the basis 
A,,A,,...,A,.. The basis of a nondegenerate program Y comprising the vectors Ay, | Ely = (Sy,Sg. - + » Sm) 
is uniquely defined, If, however, Y is a degenerate program, i.e. , some of the 


m 
A= 2s a1ji—¢; 


for j¢fy are zero, there may be several bases, It can easily be shown that the number of these bases is at 
most CY,,, where y is the number of indices jé/y for which A,;=0. There exist problems for which this 
number of bases is actually obtained (see Exercise 8), According to the definition of a pseudoprogram, to 
each basis of a support program Y of the dual problem there corresponds a pseudoprogram of the primal prob- 
lem with the same basis. In general, to a degenerate support program of the dual problem there correspond 
several pseudoprograms of the primal problem. 

We will now review the various operations involved in a single iteration of the dual method mentioning 
in particular where the assumption of nondegneracy is used. 

In the first stage this assumption is not used, In the second stage nondegeneracy of the dual problem is 
assumed twice: 

1) when computing @,, 

2) when choosing the vector A, to be introduced into the basis. 

We shall now dwell briefly on each of the above, 
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1, The parameter 6, is computed from (1.23): 


(+= min (-24) e 
x7 <0 Xi 


When the support program Y is degenerate, it may prove that for some /¢/y 
Xpj<0, and A,=0, 


and, consequently, 6,=0. If 8,=0 the new support program Y’ coincides with the previous program Y. 
The iteration, thus, merely produces an alternative basis of program Y. The linear form of the dual prob- 


lem, naturally, retains its previous value. 
In terms of the primal problem, the iteration in question transforms pseudoprogram X into pseudoprogram 


X’, Pseudoprograms X and X‘ are associated with two different bases of a given support program Y of the 
dual problem, The linear form has, therefore, the same value on these pseudoprograms, 

If @, remains equal to zero during several iterations, the process of solution throughout these iterations 
involves transformations of pseudoprograms of the primal problem corresponding to one given support program 
of the dual problem, The linear form retains a constant value throughout, and the possibility of returning to 
a pseudoprogram which has previously been examined is not excluded, i.e. , cycling may occur, 

An example of a linear-programming problem which leads to cycling when solved by the dual simplex 
method was first constructed by Beale, We shall not dwell here in detail on the occurrence and nature of 
cycling in the dual method, as we did for the simplex method in Chapter 5, $49, 

We have already noted that the solution of a primal problem by the dual simplex method is equivalent to 
the application of the simplex method to the dual problem (for details, see §8). Therefore, to each example 
of acycle in the simplex method there corresponds an example of a cycle in the dual simplex method, and vice versa. 

Indeed, suppose that cycling is detected in the solution of some problem (problem 4) by the simplex 
method, Consider the linear programming problem (problem B) which is dual with respect to problem A. 

We reduce problem B to canonical form and apply the dual simplex method. 

Pseudoprograms of problem B which correspond to the support programs of problem A constituting the 
cycle in question form a dualssimplex cycle in problem B. 

Thus, the discussion on cycling in the simplex method (Chapter 5, § 9) can be extended to cycling in the 
dual simplex method Hence, cycling may arise in the solution process of linear-programming problems by 
the dual simplex method; the minimum length of a cycle involves six iterations, 

Thus, in the degenerate case, @, sometimes vanishes, which may result in cycling. 

2. The vector A, to be introduced into the basis is chosen with the help of the formula 


— Ako, = min (-34). (4.1) 
Xrk Xey<e Xj 


We have already indicated that if the program Y’ of the dual problem obtained under elementary trans- 
formation of program Y with @=@, is nondegenerate, the index & is uniquely defined by (4.1). If, however, 
Y' is a degenerate program, ie. , 


A, =A, (8,) =A, +%x-;20 


for some j€/y, relationship (4.1), generally speaking, is not sufficient to define the vector to be introduced 
into the basis uniquely, 

Taking Ag as an arbitrary restraint vector from among the several vectors satisfying relationship (4, 1), 
may lead to cycling. If, however, the rule for choosing the vector is somewhat improved (extended), this 
vector can be uniquely defined and cycling need not be anticipated, 

4-2, Before deriving the extended rule, we shall clarify the geometrical meaning of degeneracy in terms 
of the dual simplex method, As always, we start with the first geometrical interpretation. 

Consider pseudoprogram X corresponding to a support program of the dual problem, Let the system of 
restraint vectors A; with {Ely = (8, Sy ..., Sm) Constitute the basis of program Y, and also the basis of 
pseudoprogram X. According to (2,2), the polyhedral cone Ky introduced in 2-1 lies in the lower halfspace 
with respect to the linear-form hyperplane (1.1) (IIy) passing through the point X. Again, according to 
(2,2) the condition A,=0(/¢/y) geometrically means that the edge of cone Ky formed by the intersection 
of hyperplanes (1.2) and x,==0 for jé/y, j #1, is contained in hyperplane My. Therefore, geometrically, 
nondegeneracy of support program Y of the dual problem indicates that the cone Ky and the hyperplane [ly 
have only one point in common, the point X. 

Now let the support program Y be degenerate, Consider the polyhedral set My comprising the inter- 
section of the cone Ky and the halfspace xs, «<0 (the vector Ay, is to be eliminated from the basis), Since 
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in the degenerate case some of the edges of the cone Ky belong to the hyperplane IIy, some of the vertices 
of the polyhedral set My (except the vertex X, which is invariably contained in My) will also belong to 
hyperplane ly. These vertices are, obviously, pseudoprograms of the primal problem on which the linear 
form (1.1) has the same value, 

In 2=2 we showed that the geometrical image of pseudoprogram X’ (the pseudoprogram improved in an 
iteration) is the vertex of polyhedral set My which is closest to hyperplane Ily. In this case the polyhedral 
set My has vertices (other than X) which lie on hyperplane IIy. By a successive iteration we will transfer 
from one of these vertices to another, This transformation will, obviously, have no effect on the value of 
linear form (1.1), If we choose the vertex X’ on hyperplane Hy without following certain definite rules, 
we may, after several iterations, return to the initial point X, i.e. , a cycle will form, 

Let us now give the second geometrical interpretation of the dual simplex method (see 2-7), Here the 
image of a support program Y of the dual problem is the hyperplane II spanned by the augmented basis vec- 
tors of this program, As, Asy hie Ay. Nondegeneracy of program Y indicates that for any /€/y the 
augmented restraint vector A, is strictly above the hyperplane II. In the degenerate case some of the vec- 
tors A; j¢ly may belong to hyperplane II. We have indicated in 2-7 that, + geometrically, each iteration 
involves rotation of hyperplane II about the vectors As, ; A;,, sieves As,» Asay sists » er As, until the 
hyperplane first meets one of the vectors A), where jé¢ de = (S), Sg, «+, Sm)» The rotation should be so 
carried out that the vector Ag, remains below the hyperplane, The vector Ag contained in the new hyper- 
plane II’ is substituted for the vector As, in the augmented basis of the new support program Y’, Hyper- 
plane II’ is the geometrical image of the new support program Y’, It is geometrically obvious that if Y is a 
nondegenerate program, hyperplane II’ is different from hyperplane II, iie., Y’ #Y. In the degenerate 
case it may occur that any infinitesimal rotation entailed in a given iteration will bring the vector A, (2 ¢€/y) 
to above the new hyperplane. In this case the new support program Y’ coincides with the initial program Y. 
The iteration thus merely produces an alternative basis of program Y. 

The geometrical interpretation of degeneracy not only offers a more complete understanding of this phe- 
nomenon, but also hints at the means of avoiding the dangers of degeneracy, i,e., the possibility of cycling. 
This is quite obvious in view of the first geometrical interpretation, We have seen that nondegeneracy of the 
dual problem (1,4)—(1.5), in terms of the first geometrical interpretation, indicates that any hyperplane of 
the form 


(C, X)=const 


meets at most one pseudovertex of the polyhedral restraint set of problem (1,1)-(1.3). It is geometrically 
obvious that this requirement can always be satisfied by a sufficiently small modification of the coefficients 
c, in linear form (1,1), The above is the basis from which the rule eliminating all possibility of cycling as 
the result of injudicious choice of the vector to be introduced into the basis is derived. 

4~3, We will now establish the rule for choosing the vector to be introduced into the basis, Since the 
linear- programming problem in canonical form is a particular case of a problem with bilateral restraints, 
we shall give our arguments with reference to problem (3, 1)-(3. 3). 

Let e be a positive number, 

Consider the following problem with bilateral restraints. 

Maximize the linear form 


> C;(e) x, (4, 2) 
{=1 
subject to the conditions 
n 
z A,x,=B, (4,3) 
Te By (4, 4) 
Here 
c,(e) =e, + (— 1)" ent !—i, | and Oe Sree (4.5) 
=| 1, if lejam, or xfma,, 
0, if x=B, jo=m+l. (4.6) 


The x) appearing in formula (4.6) constitute the pseudoprogram X,==(x{°), x), ,.., x) of problem 
(3. 1)-(3, 3) with the basis A,, Ag, ..., Ap. 
The preceding problem differs from problem (3, 1)-(3.3) only in the linear-form coefficients which are 
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obtained from the corresponding cy, with the aid of formula (4.5) and the parameter e. Problems of the 
form (4. 2)-(4.4) will, henceforth, be called &-problems corresponding to problem (3, 1)-(3,3), The rule for 
choosing the vector to be introduced into the basis is based on three propositions concerning e-problems. 
Theorem 4.1. There exists e,>0 such that if the vector X,=(x, x, ..., x) is a pseudopro- 
gram of problem (3.1)-(3.3) with the basis A,, A,, ..., A,, this vector is also a pseudoprogram of 
the e-problem (4.2)-(4.4) for any positive e<e,. 
Proof, Let 


A= 3 +8) Ap j=1, 2,..., a2. 


{=1 


In this case the parameters A;(e) corresponding to e-problem (4.2)-(4.4) and to the system of vectors 
A,, Ay, ..., Ag have the form 


A, (e)= 3, (e) x) —c,(e) =A;—(—1)e"+4#-/ + D (Hye tt Sl), (4,7) 
{=i 


$=1 


Let 


t= max 3x0). 


M+1Sfen fay 
We choose e, as follows 
ey=min {1, =}. (4.8) 
Then, for e<e, 
m m 
| > (—1)%4 et +4F (9) | <p etti-m > em —! 1xf%) | <cetti-™e < gn ™. (4, 9) 
tat $=1 


These inequalities follow directly from (4.8) defining e,. 
Applying equality (4.7) (for j/==m-+1) and inequality (4.9), we obtain 


>A, if v=, 
4; (e) { < Ay if v,;=0. oo) 
Since the vector X, is a pseudoprogram of problem (3, 1)—(3,3) with the basis A,, Ag, ..., A,,, then 
‘ f 20, if xf =a,, 
‘\ <=, 0, if xi) = 8B, 
(see (3. 14)). 
Hence, applying conditions (4,6) and (4,10) we obtain 
i { >A,;>0, if x=a,, 
e 
MW) < A,;=<0, if x) _B,, (4,11) 


m+lasjma. 

According to (3,14) inequalities (4.11) indicate that the vector X, is a pseudoprogram of the e-problem 
(4, 2)-(4.4). This completes the proof. 

Theorem 4,2. There exists e,>0 such that for 0 <2 <e, the dual problem of problem (4.2)- 
(4.4) is nondegenerate. 

Proof, Let A, Ay, ---- Ag, be a linearly independent system of restraint vectors, The set of indices 
of these vectors is denoted by /. The other restraint vectors A, are then expanded ‘in terms of the vectors of 
this system 


m 
Aj= py xV)As, (4, 12) 
=1 
and we take 
I . ! 
ao= Cs, (e) xe, (e). (4. 13) 


Applying (4.5), we see that A‘) (e) forany j¢/ is a polynomial of order not higher than a. The poly- 
nomial A‘) (e) % const, since it contains e*+!-/, Hence, this polynomial has at most n roots, 
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Let the least of the positive roots of the polynomial AY) (e) be nP>0. Let 
(/) qi 
Hn’ =min > 0. 
fel , / 


The parameter n' > 0 is defined by a system of linearly independent vectors A, / € The number of 
such systems is finite. Therefore 
& = min n> 0. 


Let O0<8<e,. Consider a pseudoprogram X of problem (4,2)-(4. 4) with the basis comprising the vec- 
tors Ay, | i¢€ 'x From formulas (4. 12) and (4,13), where /y has been substituted for /, we compute the 


parameters A Ux (e) of this program. 
By assumption, the least positive root of the polynomial a’ ? (e), J ¢ 7x, is on the right of e,. Hence 
NS (ec) #0 for O<e<e. 
The parameters a“e (e) ({ ¢/y) of any pseudoprogram X of problem (4,2)-(4,4) are thus nonzero, By defi- 
nition, this implies that the dual problem of problem (4.2)-(4,.4) is nondegenerate. This completes the proof. 
Theorem 4.3. There exists ¢,>0 such that if 


0<e<e,; 


and X is a pseudoprogram of the e-problem, this vector is also a pseudoprogram of problem 
(3.1)-(3.3) with the same basis. Moreover, if X is an optimal program of problem (4.2)-(4.4), tt 
Solves the initial problem (3.1)-(3.3). 


Proof, Consider a linearly independent system of m vectors Ay, {€/. From formulas (4,12), (4,13) 
we compute 


AY) = AY 0), pel. 
Let 
AM <= min fae 
(if all AY =, then A) ase), Let 


4 =min Ss) >0, (4,14) 


where the minimum is taken over the various possible linearly independent systems A, j€/. We further 
define 


m 
o = max E D | +? j < ©, (4. 15) 
t {=i 


where xi) are given by (4,12) and the maximum is again taken over the various possible systems of A, j{ € /. 
The e, specified in the theorem can now be computed from 


eyeemin {1, +\> 0, (4, 16) 


where A and o are defined in (4.14) and (4,15), respectively, 

We shall show that e, defined in (4.16) indeed satisfies the requirements of the theorem. To this end, 
taking 0<e<e,, we consider a pseudoprogram X of problem (4.2)~(4. 4). 

Let the basis of pseudoprogram X' comprise the vectors Ay, / € /y =(S,, 5 +++» Sq). | By assumption, 


(ly) 20 for x,=a, 
a"? 01 25 for igh tle (4.17) 
From (4, 7) 
Aux) (s)= = ACx — (—1)¥) et tJ 4 > (= 1" eM ttm tiglD , (4, 18) 
(=) 


It follows from (4,18) and (4, 15) that 
| ax (e)—al'x’ | <eo. 


285 


Let A;<0. Then 


In this chain of relationships, the second inequality follows from (4,14), the third inequality from the as- 
sumption concerning e, and the fourth from the definition of e, (see (4. 16)). 

Thus, for A;<0 the parameters A,(e)<0. Analogously we verify that when A,;>0, A,(e) are also 
positive, 

We can now easily show that the vector X is a pseudoprogram of problem (3,1)-(3.3) with the basis 
Ay, Agy +++ A 

Indeed, 


Smm* 
A,;=0 for xj=G;, 
since otherwise we would have obtained A,(e) <0 in contradiction to (4,17), Similarly, for x,=; 
4; <0. 

The vector X, which satisfies conditions (3,2) and (3,14), is, thus, a pseudoprogram of problem (8, 1)-(3. 3). 

To complete the proof of the theorem, it remains to show that the pseudoprogram X which is a feasible 
program of problem (4.2)~(4.4) (and thus also an optimal program of this problem) solves problem (3, 1)-(3. 3). 

From the above, X is a pseudoprogram of problem (3, 1)-(3,3). By assumption, the vector X is also a 
feasible program of this problem, Hence, according to the optimality test, the vector X solves problem 
(3, 1)-(3. 3). This completes the proof, 

4-4, We are now in a position to extend the dual simplex method to any problem with bilateral restraints, 


without the corresponding dual problem becoming nondegenerate, 
Rather than consider problem (3, 1)-(3.3), we turn to the corresponding e-problem (4,2)—(4.4) with 


0<e <e,=min {e,, &, 2e}. 


The positive numbers 8,, &, €, are specified in Theorems 4.1, 4.2, and 4,3, respectively. 

Let X, be a pseudoprogram of problem (3, 1)-(3.3) with the basis A,, Ag, ..., Ag. According to Theo- 
rem 4,1 (e <e,) the vector X, is also a pseudoprogram of the e-problem with the same basis, 

Starting with pseudoprogram X, we proceed to solve the e-problem (4,2)-(4.4), By virtue of Theorem 
4,2, this problem is nondegenerate (e<8). Therefore, after a finite number of iterations, having examined 
pseudoprograms X,, ..», Xy—y, we arrive at solution Xy of problem (4.2)—(4. 4). 

According to Theorem 4,3 (¢ <e,) the vectors X;, 1<ieg N, are pseudoprograms of problem (3, 1)— 
(3,3), the last vector in the chain Xy also being a solution of this problem, 

Thus, given a sufficiently small e, the dual simplex method applied to problem (4. 2)—(4.4) involves 
covering various pseudoprograms of the initial problem (3, 1)—(3,3) eventually obtaining the required optimal 
program. 

Let us consider this procedure in a single iteration, which involves transformation from pseudoprogram X 
to pseudoprogram X’. The vector X is a pseudoprogram of both the initial problem (3.1)—(3,3) and the e- 
problem (4,2)-(4.4). The vector X should, obviously, be considered as a pseudoprogram of the e-problem 
only while choosing the vector A, to introduced into the new basis. 

As in our discussion of the dual method with reference to problems with bilateral restraints, we again 
distinguish two possibilities: 

1, The vector Ay is eliminated from the basis, where 


In this case the index & of the vector A, to be introduced into the new basis is determined from 


Se) — min (=) (4, 19) 
Xrk =o J EE Xrj 
Here E denotes the set of indices 7 such that 
A; (e) 
f 
= >0, x 0, 
Xj ry F 


or, equivalently, such that 


134 <0 for x;=a, jeély, 
7\ >0 for x,=6,, jely. 


The basis of pseudoprogram X comprises the vectors As,, Asy .-., As. Therefore, following formula 
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(4.7), we transorm (4.19) to the form 


m v 
_ Ar (—1)%* gttin-k__ > (—1) Xih pnts 1~8;— 


*rk *rk i= =rk 
A —1y"5 my Ve 
= min Mi re | et +t~j_ py (=!) *i gns1-s, ), (4. 20) 
Je E\ Xp Mey =1 ef 


Since e is an arbitrarily small positive number, the search for the minimum in (4,20) and for the index on 

which the minimum is obtained involves comparing, successively, the coefficients preceding the various 

powers of ¢, Le., e°=1, 8, e*, .... 8%. From this we obtain the following rule for the determination of &. 
Let the set E, comprise all those indices j on which 


A 
= min (- zt) 
J€E Xj 
is obtained (comparison of the coefficients of e°=1). If &, contains only one index, it is taken as the re- 
quired k. Otherwise, the search for index k is resumed, To avoid complicated notations, we take 
S$; < 83 << 00. K Sq. 


This can always be achieved by suitably renumbering the positions of the basis. 
Let e, be the set of indices j€ E, greater than s,. Ine, we define subset @, comprising the indices 
j €e, such that 


=] yYs 
(—)) < 0. 
Xry 
If & consists of only one index, then 
k= max j. 
jee, 
If @, is an empty set, and E,=<e,, then 
k= min j. 
fee =k, 


If neither of these conditions is satisfied, i.e, , 8; is an empty set and E, #€,, we form the set £,. This set 
comprises the indices j € E, on which 


is obtained, Obviously, if e, is an empty set, no intermediate operations need intervene in the transforma- 
tion from E, to E&,. 

Observe that the rule of transformation from E, to Eg, is based on the comparison of the coefficients of 
e"+!-/J for j==s,(j € E,). If the set E, contains only one index, it is taken as & Otherwise, transforma- 
tion from E, to E, is tried, following the same rules as the transformation from E, to By. The only dif- 
ference is that the index s,_, is substituted fors,. If the set E, is insufficient to define the index k, E, 
is transformed to &,. Successive transformations either produce the index Rk or exhaust all the sets E;, which 
are, obviously, m+! in number, 

if the last set E,,,, contains several elements, the index & is defined as follows, 

We isolate the set e; comprising those indices jf € E,4, for which 


—1)¥s 
(—))” <0. 
Xry 


If €, Contains at least one element, then 
k= max j. 
J€e 


Otherwise, 
k= min je 
1 € Em+i 
This last step in the process of determining index k involves comparison of the coefficients of e*+!—/ for 
i<5,. 
The index of the vector Ag to be inuoduced into the basis is thus established in at most m-+2 steps, 
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2. We now proceed with the second case, when 
Xpy> B,. 


Here the index & of the vector A, to be introduced into the new basis is defined from 


Aa (0) min AL (4,21) 
Xe IGE Ky 


where E comprises the indices j such that 


oJ 20 for xy=a, fF ly 
Vv] <0 for xj;=B,;, 1¢/y. 


From (4,21) and (4. 7) we see that, for sufficiently small positive e, 


Vv m 
@, (ec) = min cs (=D nginsy. p> (=a erron) (4, 22) 
Xj Xpy =1 py 
is obtained on the index R, 
From (4,22) we easily derive a rule for determining k, which is analogous to what has been described 
for the first case. Since the expressions being minimized in (4.20) and (4.22) differ only in sign, we shall 
look for the index & applying the previous rule where z,; has been replaced by 


Xpy= — Xpp 


Our analysis shows that the solution process of the ¢-problem (4,2)~(4.4) with sufficiently small 8 is 
equivalent to the solution process of problem (3, 1)=-(3.3) employing the preceding rule for the determination 
of the vector to be introduced into the new basis, Application of this rule therefore guarantees that there will 
be a finite number of steps in the dual method, regardless of the properties of the problem being solved. 

4-5, To conclude this section, we retum to problem (1.1)-(1.3), written in canonical form. In this 
case a,=0, By for all fj==1, 2, ..., a. The second possibility is, therefore, unfeasible, and the set 
£ comprises the indices j such that x,,<0. We also observe that in this case vpmel for all j=l, 2, ..., a. 
Because of these features of problem (1.1)-(1.3) the rule for determining the index & takes on the following 
form, 

Let E, denote the set of indices { on which 


¢,= min (- =) 
Zej <o Xp} 
is obtained, 
If the index & has not yet been defined (£, contains several elements), we proceed with the next step, 
If all the indices jE E, are greater than sy, then 


k= min j. 
1€E, 


Otherwise, we form the set E, comprising all the indices f on which 
min 2aZ 
bas *pj 
om 
is obtained. To elucidate this part of the rule, it suffices to observe that in case of problem (1,1)-(1.3) the 
set €; always remains empty. 

If the set E, contains a single element, it is taken as the index &. Otherwise, we proceed with the next 
step, which either produces the index & or leads to a set of indices E, containing more than one element, 
The rule for transformation to £, coincides with the preceding rule, the only difference being that s,_, is 
substituted for sy, and Xm-;,¢ for xmy. We recall that according to an earlier assumption s,<S<.. 


2+ <Sm—1<Sm. The successive steps are repeated until either the index k& is determined or the set Egy, is 
obtained, In the latter case, the index k is defined as the least of ;EEp,;: 


k= min j. 
1€Em+s 


The preceding rule for determining the index & of the vector A, to be introduced into the basis eli- 
minates all possibility of returning to a pseudoprogram which has previously been examined, The rule thus 
fully guarantees against cycling. 
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§ 5. The first algorithm 


9-1. The dual simplex method, like the simplex method, can be realized 
by two different computational procedures. The difference between the 
computational procedures (the dual algorithms) lies in the different methods 
of computation of the parameters A, 

In this section we discuss the computational procedure of the first algo- 
rithm with reference to the linear-programming problem in canonical form 
and to a problem with bilateral restraints. The applications of the algo- 
rithm are illustrated by appropriate examples, 

When solving a linear-programming problem by the dual simplex method, 
the initial support program of the dual problem (more precisely, the initial 
dual basis) is assumed to be known. 

Under the first algorithm, the optimum program of a nondegenerate 
linear-programming problem is computed as follows. The basis of the ini- 
tial support program of the dual problem is taken as the basis of pseudo- 
program X of the primal problem. Expanding the constraint vector in 
terms of the vectors of the dual basis, we obtain the basis components «,, 
of pseudoprogram xX. The signs of x, enable us to establish whether the 
pseudoprogram obtained is a feasible program of the problem. If case (a) 
obtains, the initial support program of the dual problem is also its solu- 
tion, and the initial pseudoprogram is the optimal program of the primal 
problem. If some of the components x, are negative, we compute the co- 
efficients x,, in the expansion of the restraint vectors in terms of the dual 
basis. If for some x,,<0 all x,,20, the problem is unsolvable (case(b)). In 
this case the process of solution terminates by establishing inconsistency 
of the problem restraints. 

If case (c) obtains, we proceed with the second stage of the iteration, 
Using formulas (1.11), we compute the parameters A, Then we determine 
the index of the vector to be eliminated from the dual basis, and the re- 
straint vector to be introduced in its place. The new dual basis is analyzed 
in the next iteration in the same way as in the preceding step. The para- 
meters x,,x,,, and A, are computed from the corresponding parameters 
of the preceding iteration by recurrence formulas (1.12) from Chapter 5, 
as in the first algorithm of the simplex method. The computations are 
continued until the optimal program is obtained or unsolvability of the prob- 
lem is established. The solution process terminates in a finite number 
of steps. 

We see that the computational procedure of the first algorithm of the 
dual method is similar to that of the first simplex algorithm. The tableaus 
in which the iteration parameters are written are also similar in the two 
methods. 

The difference between the methods is that in one case we pass succes- 
sively from one support program of the problem to another, whereas in 
the other case from one pseudoprogram to another, In the first case we 
cover bases of the primal problem, and in the second case bases of the dual 
problem. The formal difference between the computational procedures 
lies only in the rules specifying the transformation from a given basis to 
the next one and in the optimality and unsolvability tests of the programs. 
In the simplex method, the vector to be introduced into the basis is first 
determined and then the vector to be eliminated from the basis, In the 
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dual method, conversely, the vector to be introduced into the basis is chosen 
after the vector to be eliminated from the basis has been determined. Hence 
the slight difference in the first algorithm tableaus of the two methods. 


TABLE 6,1 


é-th tableau 


a 1) 
xf | if 


Comparing the /-th tableau of the first duai algorithm (Table 6.1) with 
the corresponding tableau in Chapter 5, §2, we see that the only structural 
difference in the tableaus is that column @ is replaced by row @. Column 6 
in simplex tableaus was used to choose the vector to be eliminated from 
the basis. Row § in dual tableaus is used to determine the vector to be 
introduced into the dual basis. 

The column By, in the /-th tableau contains the basis vectors A, of pseudo- 
program xX, In column Cx we write the linear-form coefficients of the 
basis components of pseudoprogram xX, In columns A, and A, (j=1, 2, ..., 2”) 
we write the corresponding coefficients of the expansion of the constraint 
vector and the restraint vectors in termsof the basis of pseudoprogram x", 
The elements x!) in column A,=8 are the basis components of pseudopro- 
gram XW, 

The (m+ 1)-th row in the /-th tableau contains the parameters A related 
to the coefficients xf? and c, by formulas (1.11). According to the position 
in the /-th tableau, it is expedient to denote the parameter A/') as in the 
simplex method, by x, As before, 

AD gL (XM) = LO, 
The columns A,, A,, ..., A, (positions 1, 2, ..., m+1) constitute the principal 
part of the /-th tableau. The elements 6“ of the (m+2)-th row are computed 
from the formula 
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Here ris the position of the vector to be eliminated from the basis (x,.<0). 
In row 6 only those positions j, for which x,,;<0, are filled. The other 
positions are crossed out, 

The /-th iteration fills the principal part of the /-th tableau. In the first 
stage of the (/+1)-th iteration we determine which of the three cases ob- 
tains. In case (c), we proceed with the second stage of the iteration. In 
the second stage, we establish the position r of the vector to be eliminated 
from the basis, compute the row 6, determine the vector A, to be intro- 
duced into the basis, and fill the principal part of the (/+ 1)-th tableau. The 
elements of the principal part of the (/+1)-th tableau are computed from 
those of the principal part of the preceding tableau by the recurrence for- 
mulas i 
Xf a) 
iy xte for i+#r, 
x 
rk 


xf — 


yo = (1) 
= for i=r, (5.1) 
x 
rk 
i=1, 2, ..., m m+; j/=0, 1, 2, ..., 4. 


The vector A,, with the maximum (in absolute value) negative basis com- 
ponent x, is eliminated from the basis. The vector A, to be introduced into 
the basis is that on which | 


is obtained. In nondegenerate problems A, is uniquely determined from 
this condition. The rule for choosing the vector A, in the degenerate case 
will be discussed in 5-2. The basis position occupied by the vector intro- 
duced into the basis in the preceding iteration and the vector to be replaced 
are marked by arrows. The element 6, of row 4 is enclosed in a frame, 
The direction row (r) and the direction column (k) are similarly singled out. 

The (/+1)-th tableau is transformed to the (/4+2)-th tableau in the same 
way as the /-th tableau to the (/+1)-th tableau. 

The computations are checked as in the first simplex algorithm. At 
definite intervals, the parameters A,=¥x,,,,, are computed not only from 
the recurrence formulas, but also directly from (1.11). 

The bulkness of computations in each iteration in the first dual algorithm 
is approximately the same as in the first simplex algorithm. The number 
of multiplications is the same in both cases (the principal parts of the 
tableaus are transformed according to the same recurrence formulas), Di- 
vision occurs only in the computation of a and in the transformation of the 
direction row. In the simplex method the number of divisions is at most a. 
In the dual method, the number of divisions is at most 2(1—m). 

The above data, unfortunately, do not hint at the comparative difficulty 
of solving the general linear-programming problem by either of these 
methods. At present there is no statistical, and a fortiori no theoretical 
information for comparing the number of iterations required for solving 
any given problem by the simplex method and by the dual method, 

Figure 6.5 gives a block diagram of the solution of a linear-programming 
problem by the first dual algorithm. 

5-2. In our description of the algorithm, the problem was assumed to 
be nondegenerate. This assumption ensures a unique choice of the vector 
to be introduced into the basis. 
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In 4-3 we stated and proved a rule which guarantees against cycling in 
the process of solution of a problem by the dual method, Let us consider 
the application of this rule to the first algorithm. 

The restraint vectors are numbered so that the initial basis comprises 
the vectors A,, A,, ..., Ag This numeration of the restraint vectors A, is 
retained throughout the process of solution, 


aan im Case (a) 
| Stablish x,;<0 no } (optimal program) 
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FIGURE 6,5 


Now, in some iteration, let the minimum 6, in the row 6 be obtained 
simultaneously on several vectors 


Ay, dy, Ap, 
i=-— Ses =... 
~rk, *rk, + +h, 


h<hk,<... ch; 
where Xrk, <0 (é=1, 2, eae, t), 


We shall say that A,, ..., Ax, are suspicious vectors. One of these 
should be introduced into the basis of the next pseudoprogram., 

Let sp, be the highest of the indices s,, ..., s, of the current basis vectors. 
The vector Ax, is introduced into the successive basis if k>s,,for i=1, 
2, ..., # Otherwise, the set of suspicious vectors is narrowed down and in 
the next test we need consider only those vectors whose indices do not ex- 
ceed Sp. For these vectors we construct the row 


6 = ( Xp 
i Xj ad 
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The vector on which 
6,, = min6, 


is obtained (if it is unique) is introduced into the basis. If 6, is also ob- 
tained on several vectors, these form a new set of suspicious vectors. Let 
s, be the second highest index of the current basis vectors. The vector 
with the lowest index is introduced into the successive basis, if no suspi- 
cious vectors exist with indices lower than s,,. Otherwise, the set of sus- 
picious vectors is again narrowed down to comprise only vectors with in- 
dices less than s,. For these vectors we form the row 


i seat 
= ( yh! 
whose minimum elements define the next set of suspicious vectors. The 
process is continued until the set of suspicious vectors contains a single 
element. The vector to be introduced into the basis is determined uniquely 


in at most m steps. 
Consider an application of this rule. 


TABLE 6.2 


In Table 6.2 we give the coefficients in the expansion of the constraint 
vector and restraint vectors in terms of the basis of some pseudoprogram 
obtained in one of the iterations in the solution process of a problem by the 
dual simplex method. 

We see that the minimum value of the row (6,=2) is obtained simulta- 
neously on five vectors: 


A,, Ay, Ay Ags Ais: 
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In this case r=3, so that t=— se. Among the basis vectors A, =A,, is the 
$ 


one with the highest index. Vector A,, is, therefore, eliminated from the 
set of suspicious vectors. The restraint vector A,, occupies the fourth 


position in the basis. We compute the ratio a (elements of the row §6,) for 


the remaining suspicious vectors, The least value in the row §, is ob- 
tained on the vectors A,, A,, and A,. 

The second highest index of basis vectors is s,=10. The indices of all 
the remaining suspicious vectors are smaller than 10. We compute the 


elements of the row 6,= (=) for j/=1, 6, 9. All three 6,-values are equal. 


Continuing with this procedure, we eliminate the vector A,, since its index 
is higher than the third highest index of the basis vectors, A;,=A,. We are 
left with two suspicious vectors A, and A,. Finally, the row 4, enables us 
to establish the index of the vector to be introduced into the basis, This 
is A,, for which 


Au +) = min (3, 2)=2 
J 


X36 


is obtained. 

The dual method can, thus, be equally well applied in degenerate and 
nondegenerate cases. Degeneracy of the dual problem only complicates 
the choice of the vector A, introduced into the basis. However, the number 
of iterations always remains finite though, generally speaking, the linear 
form does not decrease monotonically. 

It is noteworthy that when solving applied problems by the dual method, 
as in the case of the simplex method, cycling is most improbable. More- 
over, construction of examples illustrating cycling requires a certain in- 
genuity. Also the method for choosing the vector A, which guarantees 
against cycling is comparatively complicated. Itis, therefore, advisable 
in practical computations to use certain simpler rules for choosing the 
vector to be introduced into thebasis., One of these rules states, for 
example, that the suspicious vector with the lowest (or the highest) index is 
introduced into the basis. In most cases this elementary rule is sufficient 
to prevent returning to a basis which has previously been examined. In 
those rare cases when this simplified rule produces cycling, we should ap- 
ply the exact rule when the same basis recurs, The exact rule is applied until 
we obtain a pseudoprogram basis which actually decreases the linear form, 
We then return to the simplified rule which specifies uniquely the vector to 
be introduced into the basis. 

It should be kept in mind that in each transition from the simplified to 
the exact rule, the restraint vectors should be renumbered so that the basis 
in question comprises the vectors A,, A,, ..., A,, and this numeration should 
be observed at least until the linear form decreases for the first time. 


5-3, We shall illustrate the solution of a linear-programming problem by the dual method with two 
exainples, 
Example 1. Maximize the linear form 


L (X) = — (8x, +4, + 2x, + 3x, + x, + 2x, + 5x,) 
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subject to the conditions 


2x, + 3x, +2x, —x, =10, 
4x, +3x,+x, —X, = 4, 
x, +52, + 4x, +5x, —x,, =2, 
3x, +2x, +3, —X;, == §, 
c+ Qxs+ 4x, +3x, —x, ~=3, 


x0; /=1, 2, ..., 12. 


Solution. The dual problem is stated as follows: 
Minimize the linear form 


L(Y) = 10y, + 4y, + 2y, + 5y, +3y, 


2y, +3y = —3, 
4y, +y, = —l, 
34 +5y, +2y, > —2, 
24, + 44s = —3, 
3y, +4y; = —Il, 
24, +H +3y, = —2, 
Sys +3y, > —5, 

y<=0, i=l], 2, 3, 4, 5. 


subject to the conditions 


We see that Y=(0, 0, 0, 0, 0) is a support program of the dual problem, The vector Y reduces the 
last m=5 restraint to equalities: 
yj =0; 
the other restraints of the dual problem are reduced by Y to strict inequalities. The basis of the support pro- 
grain Y of the dual problem comprises the restraint vectors 


A,=(—I, 0, 0, 0, 0), A,=(0, —t, 0, 0, 0), 
Ay =(0, 0, —I, 0, 0), A, =(0, 0,0, —1, 0), A, =(0, 0, 0, 0, —!). 


All the computations entailed in the solution of Example 1 are given in Table 6.3 (6-3). 

The basis of the initial pseudoprogram comprises unit vectors with reverse sign. The basis components 
of pseudoprogram X (zeroeth tableau) therefore coincide with the corresponding components of the constraint 
vector taken with reverse sign, and the elements xj of columns A; are equal to the elements of the cor- 
responding columns of the restraint matrix || a;,|| with reversed signs. In the (m-+1)-th (sixth) row of the 
tableau, we write the parameters A,;=—c, (the basis components of the initial pseudoprogram correspond 
to zero coefficients of the linear form). 

All the basis components of the pseudoprogram are negative and in each row there are some negative 
xj; ({==1,2,3,4,5). This indicates that we have case (c). We now pass to a new pseudoprogram, The vector 
As, =A, (%,=%,,.= —10<x;,, =2, 3, 4, 5) is to be eliminated from the basis, In the last row of the 


zeroeth tableau we write 


for %;<0. The least element §@, of the row @ is obtained on A,, The restraint vector A, should be intro- 
duced into the pseudoprogram basis in place of the vector A,, The principal part of the zeroeth tableau is 
transformed into the principal part of the Ist tableau using recurrence formulas (5. 1). 

The Ist tableau and computations connected with transformation to the 2nd tableau are analyzed ana- 
logously. The pseudoprogram obtained after the third iteration proves to be a feasible program of the problem. 
The problem is thus solved by the vector 

X’’’ =(0, 0.583, 2.222, 0, 0, 1.667, 0, 0, 0, 9.694, 0, 1.444). 
The maximum of the linear form is equal to 
L (X’"’) = — 8,361. 


In Tables 6.4 (0-2) we give the sequence of solution of another example, 
Fxample 2, Maximize the linear form 


L (X) = — (5x, + 3xq + 4x5 + x4 + 2%, + 3x,) 
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subject to the conditions 


6x, +2x,+3x,4+8x,4+ x,+2x,—x, = 12, 
4x, +5x,+8x,+3x,+ x,+7x, —x, = 5, 
2x, + %+4x,+ 2x, —x, = 7, 
OX, + We, Xe x +6x, + 5x, —Xyo =10, 
x, +3x, + 6x, +3x, + 2x, —*%, = 4, 


xj;220; j=1,2,..., 1 


It is unnecessary to explain the tableaus, 

5-4. We now present the dual simplex method (or, more precisely, its 
first algorithm), without resorting tothe linear-programming problem in 
the vector form and without introducing the dual problem. 

We shall limit the discussion to the nondegenerate case. 

We solve the problem restraints (1.2) for the m variables x,, ..., Xs, 
corresponding to the basis components (x,,, ..., x,.) Of the initial pseudopro- 
gram X. As previously, let / denote the set of indices of the basis vectors, 

We have seen in Chapter 5, § 4, that the linear-programming problem 
(1.1)—(1.3) can be stated in the following equivalent form. 

Compute the nonnegative x,,x,,...,.x, satisfying the restraints 


Xs; = Kiam Be itp i= 1, 2, ooey mM, (5.2) 
and maximizing the linear form 
L(x) =L(X,)— 3% A,x,, ses 
( ) ( o) p> iat ( ) 
where X, isthevector whose components x, are equalto «;, (i=1, 2, ..., m), 
and the cther components areallzero. xX, is apseudoprogram of problem 
(1.1)—(1.3). This indicates that the choice of variables x,, ..., xs;, ensures 


nonnegativity of all the coefficients of A, inexpression (5.3) of linear form L(X). 
Let 


L(X)=Xsqgir 2 (X.)=Xaas,o Aj=Xmsif (5.4) 


and consider the system of equations combining equalities (5.2) with (5.3): 
fu= Kia Jy isp i= 1, 2, oe | mM, m-+-i1, (5.5) 

We solve this system by the Gauss method (complete reduction method). 

We express x (k¢/) from the r-th equation (!<=r<™m) and insert the result 


inthe other equations (5.5). It is naturally assumed that x,.0. After the 
first step of the complete reduction method, we obtain 


X= Xe— Dy X1jX py = 1, 2.0 way MP, m+, (5.6) 


{sare SA poe f 


The parameters x, are determined from 


Xik 
; *ij— 2 eh ir, 
xi= 


, a 
where s=s, for ir, s,=k; 


rf (5.7) 
Xr 

i=1, 2, soo, M, m+- 1; jJ=0, 1, 2, coo, A, 
The element x,, is called the direction element of the transfor- 


mation. 


» fxr, 
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Until now we have imposed no restrictions on the choice of the number 
r of the equation and the index & of the variable specifying the direction ele- 
ment of the transformation, except for the obvious requirement x,,40, We 
shall now impose the following restrictions on the choice of rand &., 

The index r is specified by the greatest (in absolute value) negative x,, 


Xpy= mun X19 


(if all x,,>0, the program is optimal). 

Now, there exists at least one feasible program of problem (1.1)—(1.3) 
if for x,,<0 at least one of the x,,(/=1, 2, ...,”) is negative. Otherwise, as 
we see from (5.6), 


a’ 
x, = Xp <0 


for any nonnegative extrabasis variables x,(/¢ J). 
The choice of the index & should ensure a transition to a new pseudo- 
program X’, This indicates that for the given k 


A)= Xm, |=, {j= 1, 2, eee, ni, (5.8) 


According to recurrence formulas (5.7), condition (5.8) can be rewritten in 
the form 


Aj = A,—5* x,,>0, 7 ee (5.9) 


The vector X, is by assumption a pseudoprogram, Hence, 
A,=0, jJ=1, 2, ee ty n, 
For j/=s, x,,=1, Ay=0. Hence, from (5.9), 


Ay 
ge 0. 
But A,>0 (a nondegenerate case is being considered), Therefore x,,<0, 


f now 
: x,,>0, 


(5.9) is automatically satisfied. 
We have seen that a solvable problem must contain at least one x,,<0. 
To satisfy condition (5.9) it is necessary that for x,,<0 


i,e,, the index & is specified by the variable x, for which 


Ap _ A; 
— zt min (—3). (5.10) 
We now compare systems (5.5) and (5.6) with the tableaus of two successive 
iterations of the first dual algorithm. We see that the elements of the right- 
hand sides of the equations are arranged inthe same order and are trans- 
formed according to the same recurrence formulas as the corresponding 
elements in the tableaus. 

The dual simplex method, with a given initial pseudoprogram, thus re- 
duces to solution of systems of equations by the Gauss complete reduction 
method with special rules imposed on the choice of the direction element x,,. 
The choice of r is restricted by 

Xp_= Min Xjq, 
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and the k chosen should satisfy (5.10). In other words, the choice of the 
direction row ensures monotonic variation of the linear form, and the 
choice of the direction column, motion over pSseudoprograms. 

5-5. Let us now consider the specific features appearing when the first 
dual algorithm is applied to problems with bilateral 1 estraints. 

The modifications introduced in the computational procedure because of 
the bilateral restraints apply mainly to the method of determining the direc- 
tion element of the transformation and to the formulas for computation of 
the basis components of the pseudoprogram,. This also causes some modi- 
fications in the structure of the first-algorithm tableaus, 

The /-th tableau corresponding to the /-th iteration in a problem with 
bilaterally restrained variables differs from the /-th tableau of the problem 
in canonical form in the additional row X and in three additional columns 
A,, (a: 6), and 6 (Table 6.5). 

In row X, under each of the restraint vectors not appearing in the basis, 
we write a, if xj? assumes the left boundary value (i.e., if A,>0), and if 
xj? is equal to the right boundary value (i.e., if A,<0), The cells of row X 
corresponding to the basis vectors are crossed out, 

To the right of the tableau columns corresponding to the restraint vec- 
tors we add two columns, (a; f)y and 6, In column (a; f)y we write the 
boundary values for each of the basis variables, The entries in column 6 
are equal to the deviations 6” of the basis components of the pseudoprogram 
from the boundary values of the corresponding variables: 


(é) : (6) 
6{? -{ a, —Xio ’ if Xio <ds, 


5.11 
xQ—By,, if x>Bs,. ( ) 


The cells of column 6 for which 
a, xi) < By, 


are crossed out, The numerical values of 6! should be modified by a or 
B depending on whether -x,, is on the left or the right of the permissible 
interval of the corresponding variable. 

The vector A,;, corresponding to the maximum deviation 8 should be 
eliminated from the basis. 

The modification a or f added to 6!” indicates which of the cases, (i) or 
(ii), applies for the given iteration (see 3-5). 

In (i), the elements of the last row § are computed from 


ij =——,. (5.12) 


Actual entries are made only in those cells of the row § for which 


A‘! 
— <0, x 0. 
x) Te 


The remaining cells of the row 6, as those corresponding to the basis vec- 
tors, are crossed out, In the degenerate case, when A,=0 from some A, 
not appearing in the basis, we must treat A, as a positive number for x,=a, 
and as a negative for x,=B,. 

In(ii) the entries in row § are computed from 


() 
5) (5.13) 
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Actual entries are made se in those cells where 


“>0, xh) £0. 


The other celis of the row § are crossed out. Asin (i), the zero A, arising 
for extrabasis variables in the analysis of degenerate programs are treated 
as positive for x,=a, and as negative for x,=6,. 

The column A, contains the basis components of the pseudoprogram (the 
coefficients in the expansion of the vector 


A, = B— x 
° Ry yA; 


in terms of the basis vectors of the pseudoprogram), The vector A, changes 
from one basis to another and, therefore, the general recurrence formulas 
used in transforming the tableau do not apply to it. However, it can be 
verified that the basis components of the pSeudoprogram in the (/+1)-th 
iteration are related by recurrence formulas (5.1) with the components of 
the vector 


Ay = Ay + xh Ag— YP Aas (5.14) 


corresponding to the /-th iteration. Here s, is the index of the vector A,, 
eliminated from the basis in the /-th iteration, k is the index of the vector 
introduced into the basis inthe /-th iteration; y!” is equal to a,, or 6,, de- 
pending on the modification (aor f) of the deviation 6”. 

The column A, is inserted into the tableau between columns A, and A, 
The elements of column A, are computed from 


1 ty od 
~ (ty -| xO + xp xf? for /#r, 


a 4 gh? — y for t=r, (5.15) 


The columns A,, A,, ..., A, (all (m+1) positions) constitute the principal 
part of the tableau. 

The initial zeroeth tableau differs from all the subsequent tableaus in 
having two additional rows. In row C we write the linear form coefficients 
of the problem, and in the row (a; f) the boundary values of the variables x, 

Let us now briefly outline the sequence of computations in a single 
(¢+ 1)-th iteration, 

At the beginning of the (/+1)-th iteration the /-th tableau is assumed to 
be filled (except for the column A, and the row @). The iteration starts by 
testing pseudoprogram X for optimality. Pseudoprogram xX is a feasible 
program and thus a solution of the problem (case (a)) if all the positions in 
column 6 of the /-th tableau are crossed out. If (case(a)) does not apply, 
we should check for solvability of the problem, The problem is unsolvable 
(case (b)) if there is a row with a deviation 6{) (a) satisfying condition (3.19) 
or a row with a deviation 6" (f) satisfying condition (3.20). In other words, 
the problem is unsolvable if in a row with a deviation modified by a the 
elements x,,;20 for x,=a,and x,,<0 for x,=§, (/€/,) or in a row with a devia- 
tion modified by B the elements x,,<0 for x;,=a,and x;,,20 for x,=6,(/€/)). 
Here x; are the elements specified by the row X of the /-th tableau. 

If the unsolvability test is negative (case(c)), we determine the direction 
element of the transformation and construct the (/4+1)-th tableau (except for 
column A, and row 6), containing the initial data for the next iteration. 
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The vector A;, with the maximum deviation 8 is eliminated from the 
basis. If there are several such vectors, any of them can be eliminated. 
To determine the vector to be introduced into the basis, we fill the row 
according to the preceding rules, The least element 6, of the row 6 speci- 
fies the vector A, to be introduced into the basis. If 9, is simultaneously 
obtained on several elements, the vector with the lowest index is introduced 
into the basis, The exact rule for choosing the vector A, which guarantees 
against cycling is given in 4-4. 
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Given the index of the direction row r and the index of the direction 


column k, we may compute the elements of the column A, from (5.14) or 
(5.15). We now have all the data for passing to the (/+1)-th tableau. 
The columns 4A,, A,, ..., A, (positions 1, 2, ..., m, m+ 1)are transformed 
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according to the recurrence formulas 


* Vo (6.16) 


The elements of column A, of the (/+1)-th tableau are computed with the 
aid of the same recurrence formulas from the entries of column A, in the 
i-th tableau: 


~«C p 4 
Xig — TP Kips i-r, 
xfit) Xr 
: rau (5.17) 
“H i#r. 
x rk 


The value of the linear form is transformed between two successive itera- 
tions according to the formula 


L(X¢*9) = LX) — 008)", (5.18) 


The entries in row X in the (/+1)-th tableau coincide with the correspond- 
ing entries of row X in the /-th tableau for all j, except for j=k and j=s,. 
The j=& position of the row X is crossed out (the vector A, appears in the 
basis of pseudoprogram X“+"), In the s,-th position the modification (aor 
B) of 5!’ from the /-th tableau is written. 

In column (a; B)y of the ((+1)-th tableau we write the same figures as in 
column (a; Bf), of the /-th tableau, The only exception is the r-th position, 
where (a;,; B,,)iS substituted for (a,; B,). The column 6 is filled according to 
formulas (5.11) following the preceding remarks, 

Determination of all these parameters completes the (/+1)-th iteration. 
Subsequent iterations follow the same rules. 

The computations should be checked by determining A, and L from re- 
currence formulas and directly from the relationships defining these para- 
meters. 

Figure 6.6 is a block diagram of the solution of a linear -programming 
problem with bilateral restraints by the first dual algorithm. 

5-6. We will now illustrate the application of this computational procedure by an appropsiate example. 

Determine the vector X maximizing the linear form 


L=12x, + 2x, + 10x, + 14x,+%, +22, 


subject to the conditions 


2x, + x,+3x,+5x,4 x, =|, 
%, + 4x, + 2x, + 3x, +X, = 4, 
4x, + 2x, + 3x, + 5x, +x, =x 4, 
3x, + 2x, +4x,+ x, +%,=8, 


—lexz,sl, j=}, 2, oeeg 8. 


The process of solution of the problem is given in Table 6.6 (0-2). All the variables are bilaterally 
restrained, Therefore, any system of m=e4 linearly independent vectors can be taken as the basis of the initial 
pseudoprogram. It is natural to choose the unit vectors A,, A,, A,, and Ag as the initial basis. In the 
columns A,, A;,..., A, (the first m=4 positions) of the zeroeth tableau the components of the restraint vec- 
tors are written. The elements of the row A are computed from (1.11). In row X we write the extrabasis 
variables. For A,>0, x,=a,, for A;<0, x;=B, Here 


%=cQ,e—1, x,=P,—x,=p,=%,=B,=1. 
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In column A, we write the basis variables of the pseudoprogram 
A,=B— >> x ;A j. 
i¢ly 
We have, for instance, 


Xgq = by — (PB Gq, + OgGee + ByGeg + Bay.) =4—(1-1—1-44+1-241-3) =2. 


The value of the linear form on the pseudoprogram is equal, by definition, to 
| | 
L (X) = » cx = 12-1 +2-(—1) +10-1 + 14-1 +0-(— 8) + 12-+2-(— 6) +0-2=24, 
=1 


The elements of column 8 are computed from (5.11). Column 6 contains some entries. Hence, the initial 
pseudoprogram does not solve the problem. The unsolvability test is also negative. We have thus case (c). 

The vector A, =A, corresponds to the largest deviation 6,=7 and so should be eliminated from the 
basis. 6, is modified by a. The row 6 is, therefore, computed from (5.12). Only the three positions cor- 
responding to the vectors A,, A,, and A,, for which 


A A; 
xpi Fay 


are filled inthe row. The least value 9, in the row 4 is 0.2, which is obtained on the vector A, The re- 
straint vector A, should thus be introduced into the successive basis. The direction row of the transformation 
is thus the first row (r=1), and the direction column is column A, (&==4). We are now in a position to 
compute the entries of column A, from (5.15) and to passtothe Ist tableau. The elements Xi (i=1,..., 5; 
j=1,..., 8) are computed from recurrence formulas (5.16). The basis components of the successive pseudo- 
program are determined from (5.17) using the elements of column A,. The linear form on pseudoprogram 

X’ is equal, according to (5.18), to 


L (X) =24—0.2-7 =22.6. 


In row X the cell corresponding to A, is crossed out, and in the position corresponding to Ay, we write 
the modifier a@ accompanying the deviation 6, in the zeroeth tableau. In all the other positions in row X 
the same modifiers as in the zeroeth tableau appear. 

Analysis of pseudoprogram X’ and determination of the parameters of the successive pseudoprogram 
follow the same rules, Observe that in the first iteration we have (i), and in the second case, (ii). 

In the 2nd tableau all the cells in column 6 are crossed out. Pseudoprogram X” thus solves the problem: 


X" =(1; 0.529; 1; —0.706; —1; 1; —0.529; 0.647); 
L (X”) = 13.118, 


§ 6. The second dual algorithm 


6-1. The second dual algorithm differs from the first algorithm in the 
Same aspects in which the second simplex algorithm differs from the cor- 
responding first algorithm, 

Let us first apply the second algorithm to the canonical form of a prob- 
lem, 

The solution of the problem starts with some given initial support pro- 
gram of the dual problem, The computations are arranged exactly as in the 
second simplex algorithms, i.e., in a series of principal tableaus and one 
auxiliary tableau. 

In the principal tableaus (Table 6.7) we write the basis components e‘) 
of the pseudoprogram and the coefficients e{) in the expansion of the m -di- 
mensional unit vectors 


e,=(0,...,0, 1, 0, ..., 0) 
m 


in terms of the vectors of the dual basis. The (m+ 1)-th row of the principal 


308 


tableau contains the value of the linear form 


m 
cd { 
nek: e = 2 biyt 
st 
and the components 


(f) (t) 
eCm+1, 6 = VM 


of the support program of the dual problem, If only the initial dual basis 
is given, y, are computed from the formula 


Yj=Cmar f= aaeatip — J, 2, ey (6,1) 


The columns e,, e,,..., ¢, (all (m+1) positions) constitute the principal 
part of the tableau. When passing from a given tableau to a succesSive one, 
the principal part of the tableau is transformed according to the same re- 
currence formulas (5.5)—(5.6), Chapter 5, as the elements of the principal 
tableaus in the second simplex algorithm. 


TABLE 6.7 


The principal f-th tableau 


e | e; e; | em | A) 
(4) (2) (hy (1) | (2) 
C10 ey Crs ey | Xik 
78) TO) (ht) (ft) (i) 
#90 Cr Ces Com || *ak 
(4) a (4) fy (t) 
ero e; : ers om Xrk 
@) () (i) (ly 1 
emo emi ema ée 1 xt4 


Observe that in the second simplex algorithm the (m+ 1)-th row is re- 
quired for subsequent computation of the parameters A" in each iteration, 
In the second dual algorithm, the parameters A"), as will be shown in the 
following, are computed from recurrence formulas. The elements e,,,,;=y; 
are, therefore, required only in the zeroeth tableau for computing the initial 
A,. In all the other principal tableaus, the (m+ 1)-th row is of secondary 
importance, as it is used for computational checks. 

The last column (4{”)of the principal tableau contains the coefficients «!? 


in the expansion of the vector A, to be introduced into the basis in terms of 
the basis vectors of the pseudoprogram, The components x‘) are computed 
from 


m 
Xie = Dd eteayy. (6.2) 
$=1 


The entries of column A, are required for transforming the principal part 
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TABLE 6.8 


Auxialiary tableau 


2154 
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of the tableau according to the recurrence formulas 


ij 7 rir t#I, 
elit?) ott . (6,3) 
la 
’ i= ’ 
y ; 
i=1, 2, ee oy m, m+; J=0, 1, 2, ooo, M, xO = AP. 


The upper part of the auxiliary tableau contains the restraint vectors. In 
the (m+ 1)-th row we write the linear-form coefficients, 
To each principal tableau correspond three rows in the lower part of the 


auxiliary tableau (Table 6.8): the row X the row A!) and the row 6, 


The entries of the row A! are computed from 


m m 
A} = Lay —e,= Dajyemes.1—cy (6.4) 
=1 = 
for /=0. The parameters Af of subsequent iterations are determined re- 
currently. 
In the row X*” we write the coefficient «‘) in the expansion of the vectors 
A,in terms of the dual basis, corresponding to the vector A,, The coeffi- 


cients x‘? are computed from 


xr} = Ben ap J2 1,2, 2000: (6.5) 
—# | 


Here r is the basis ‘position from which a vector is eliminated in the 
(i+ 1)-th iteration. 
In the row 6 entries are made only in those positions for which x‘)<0, 


Each entry of the row 6“ is equal to minus the ratio of the entries in the 
t) 


two preceding rows. The least entryinthe row 6, equal to — [>> is denoted 
by 6). The vector A,, on which 6{?is obtained, is introduced intoAne Bucs 
cessive basis, 

If 0, is obtained simultaneously on several vectors A,,, any of these can 
be introduced into the basis, e.g., the vector with the lowest index. How- 
ever, this rule for choosing the vector to be introduced into the basis does 
not guarantee against cycling. Whenacycle is detected, i.e., whena 
basis which has previously been examined comes up again for consideration, 
we should choose the next vector to be introduced into the basis according 
to the exact rule discussed in § 4. The application of this rule to the 
second algorithm does not differ from its application to the first dual al- 
gorithm. However, in the first algorithm, the coefficients x,, (see 5-2 
where the application of the rule is illustrated) are contained in the tableau 
proper, whereas in the second algorithm the required parameters x,, have 
to be computed from formulas (6.5). This fact complicates the application 
of the rule to guarantee against cycling when solving problems with de- 
generate programs by the second dual algorithm. 

Computation of the row X’is the most complicated operation under the 
second dual algorithm, The determination of each entry x‘) requires m 
multiplications. However, as we have already indicated, the introduction 


of row X;) eliminates the necessity of computing the parameters A, from 
cumbersome formulas (6.4), Indeed, according to recurrence formula (5,1), 
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we have for i=m+1 (x,,,,,;=A,) 


I+1) a) xy (!) a3 (6.6) 
Ajit) = Af ——y AW, f=0,1, 2.0.5 
rk 


or, in our notations, 
AV*Y = AY 4 6x9, f= 0, 1, 2, 0005 (6.7) 


i,e,, the row A“+” is obtained as the sum of row A“ and row Xf” multiplied 
by the chosen element 6!” of row 6, 

6-2, Let us now briefly outline the sequence of operations under the 
second simplex algorithm, 

The computational procedure of the second algorithm involves successive 
iterations carried out according to the same rules. Each iteration consists 
of two stages. In the first stage, the pseudoprogram is tested for optimality 
(case(a)). Ifthe pseudoprogram is not a feasible program of the problem, 
test for unsolvability (case (b)). This completes the first stage of the itera- 
tion. If case (c) obtains, we proceed with the second stage, where we com- 
pute the successive pseudoprogram and all its characteristic parameters, 

Consider the sequence of computations in the (/+1)-th iteration. Testing 
pseudoprogram X” for optimality, we examine the signs of the entries in 
column e, of the principal /-th tableau, If all the entries ef? (i=1, 2,..., m) 
are nonnegative, the pseudoprogram is a feasible program and simulta- 
neously a solution of the problem, Now let some of the ef) be negative. The 
vector A, with the greatest (in absolute value) negative component e'? is 
chosen as the vector to be eliminated from the basis. 

Next we compute the entries of the rows A“ and X of the auxiliary 
tableau. The row Ais filled according to recurrence formulas (6.7) from 
the parameters of the three preceding rows (A*-”, Xi!" and 6¢-), 


We should emphasize that the indices r in X""and x correspond to 
different positions of the dual basis. In the former case, r is the basis 
position from which a vector is eliminated in the /-th iteration, and in the 
latter case, the index r indicates the basis position from which a vector is 
eliminated in the ([+1)-th iteration. The exact notations Xi? and X7) would 
have unnecessarily complicated the formulas. 

The entries x) of the row X are computed from (6.5) as the products 
of the r-th row of the principal /-th tableau by the corresponding columns 
A, from the upper part of the auxiliary tableau, The row X!” enables us to 
test the problem for unsolvability, The linear-programming problem (1.1)- 
(1.2) has no feasible programs, if all the entries in row xX!” are nonnegative 
(case (b)), 

Note that under the second algorithm unsolvability of the problem, if it 
is indeed unsolvable, is established after more iterations than under the 
first algorithm. This is due to the fact that under the second algorithm we 
compute not all the coefficients x,,, as in the first algorithm, but only those 
x,, Where r is the index of the greatest (in absolute value) negative com- 
ponent of the pseudoprogram., 

Case (c) obtains if the row xX!” contains at least one negative element. In 
this case, we proceed with the second stage of the iteration, i.e., choose 
the vector A, to be introduced into the basis, expand it in terms of the basis 
of pseudoprogram xX” and compute the main part of the (/+1}th principal 
tableau. 
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To choose the vector A,, we compute the row @ of the auxiliary tableau. 
As we have already indicated, the entries 


are determined only for «)<0, The vector A,on which 6), the element 


with the least value in the row 6”, is obtained is then introduced into the 

basis. To ensure unique choice of A, in the degenerate case we should fol- 

low the. simplified or the exact rule, according to the considerations in 6-1. 
In the last column Aj) of the principal /-th tableau we write the coeffi- 


cients x!) in the expansion of the vector A, in terms of the basis vectors of 


pseudoprogram xX”, The parameters x‘? are computed from formulas (6.2) 
as products of the i-th row from the main part of the /-th principal tableau 
and the &-th column from the upper part of the auxiliary tableau. 

We now have all the necessary data for transforming, with the aid of 
formulas (6.3), the main part of the /-th principal tableau into the main part 
of the (/+1)-th tableau. In other words, we now have all the initial para- 
meters for the (/+2)-th iteration, 

The process of constructing the principal tableaus and extending the 
auxiliary tableau produces after a finite number of iterations the optimal 
programs of the dual pair of problems or else establishes their unsolvability. 

Computations under the second dual algorithms can be carried out in two 
possible ways. The first of these is based on the simultaneous computation 
of the parameters 


Cnsi, =I (j=0,1, ..., m) 


from recurrence formulas (6.3) and directly from (6.1). 

Checking in the above case is carried out by computing A, from (6.4) and 
from (6.7). 

In Figure 6,7 we show a block diagram of the solution of a linear-pro- 
gramming problem by the second dual algorithm, 

6-3, The computations of a single iteration of the second dual algorithm 
are slightly more cumbersome than those of a single-iteration of the second 
simplex algorithm, When compiling the principal tableaus in the dual 
method, the column @ is not computed, Moreover, we have m multiplica- 
tions less by not computing the (m+1)-th row in the principal tableau. On 
the other hand, in each iteration of the dual method the auxiliary tableau 
is extended by three rows, instead of one in the second simplex algorithm, 
Computation of the row A“ from recurrence formulas requires nzn—m multi- 
plications (the A, for the basis vectors areall zero), Compilation of the 


® entails (n—m)m multiplications (the (x9 for the basis vectors are not 


row X 

computed; x?=1, and x!}=0for /=1, 2, ..., m; i#r), The same number of 
multiplications was needed inthe second simplex algorithm for computing 
A,. Finally, the row 6 is computed with less than (a—m) divisions. 

Note that in the simplex method thereis certain freedom in the choice of 
the vector A, introduced into the basis, The vector A, can, generally 
speaking, be taken as any vector A, with a negative evaluation A, There- 
fore, in the second simplex algorithm we may economize on multiplications 


by computing the parameters Aj’ until the first negative evaluation appears, 


Aj? <0. In the dual method (the nondegenerate case), the vector A, is 
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uniquely specified by the least element (0° in the row 6”, To form the row 
4s“ we must compute all the elements of rows Aj’ and xX; From the above 
we see that in a single iteration the second simplex algorithm has some 
advantage. 
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FIGURE 6.7 


As we have already indicated in our discussion of the first algorithm, 
no evaluations are available for comparing the total number of iterations 
required for obtaining a solution by the two methods. 
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Comparing the first and second dual algorithms we reach the same con- 
clusions as when comparing the two simplex algorithms. 

6-4, Let us now illustrate the solution of a linear- programming problem by the second dual algorithm 
with the example considered in 5-3 in connection with the first algorithm. 

The entire computational procedure is shown in Table 6.9 (principal tableaus 0-3) and in Table 6, 10 
(auxiliary tableau). 

As the initial dual basis we take the same system of vectors as under the first algorithm, The basis 
variables do not enter the linear form of the problem, The elements in row A of the auxiliary tableau there- 
fore coincide with the corresponding linear-form coefficients, taken with the reverse sign. The vector 
As,=A, corresponding to the greatest (in maximum value) negative pseudoprogram component is eliminated 
from the basis, 

The row X,, whose entries are computed from formula (6.5), contains some negative values. Hence, 
case (c) obtains, 

In row @, only the positions 1,3, and 6 corresponding to negative entries of the row X, are filled. The 
least element of the row @ corresponds to the vector A,, This vector should be introduced into the first basis 
position in place of A,. The expansion coefficients of A, in the initial basis are computed from (6,1) and 
written in column A of the principal zeroeth tableau, In the right bottom corner of the tableau we write 
A,=A,=2. Then, the main part of the zeroeth tableau is transformed into the main part of the principal 
lst tableau by means of the recurrence formulas 

’ | 
"7 th 
e; prey Cy %ias 
i=!, 2, 3, 4,5, f=0, 1, 2, 3, 4, 5. 


For example, | 
P ey =—z + (— 1) =0.333, 


es, =0—0.333-(— 2) = 0.667. 
We now proceed with the first stage of the 1st iteration. 


The column e, again contains negative entries only, The vector A,s,=A,, should be eliminated from 
the basis. The row A’ofthe auxiliary tableau is computed from recurrence formula (6.7). We have, e.g. , 


A, =A, + 95x,,= 2 + 0.667 -(— 2) =0.667. 


The row X, of the auxiliary tableau has some negative entries, Thus, we need not conclude that the problem 
is unsolvable, 

We now proceed with the second stage of the Ist iteration, We fill the row 6’. The least element of 
this row 0, == 0,222 is obtained on A,. The restraint vector A, is introduced into the tourth position of the 
successive basis in place of As,=A,,, In the last column Af’ of the principal Ist tableau we write the coef- 
ficients ae ot the vector A, in the current basis, as computed from (6.2). The (m-+1)-th (sixth) entry in 
column Af? is A, =0.667. 

The principal 1st tableau is, thus, completely filled and ready for the transformation of its main part 
into the main part of the principal 2nd tableau. Continuing with this procedure we obtain after the third 
iteration pseudoprogram X*’ with positive components, The program X”*’ solves the problem. The basis 
components of. the optimal support program of the problem are written in column e,of the principal 3rd 
tableau, The maximum of the linear form of the primal problem and the components of the solution vector 
of the dual problem are written in the last row of the principal 3rd tableau. 

In Table 6.11 (principal tableaus 0-2) and in Table 6.12 (auxiliary tableau) we list the sequence of com- 
putations under the second dual algorithm for Example 2, considered in 5-3 in connection with the first algorithm, 


6-5, Let us now consider some features of the application of the second 
dual algorithm to linear-programming problems with bilateral restraints, 

Taking into consideration the remarks in 5-5, we may, without difficulty, 
modernize the computational procedure of the second dual algorithm and 
adapt it to the solution of linear-programming problems with bilateral re- 
straints. 

The principal tableaus of the second algorithm for a problem with bi- 
lateral restraints differ from the principal tableaus for a problem given in 
canonical form in having three additional columns, specifically (a; B)x, 6, 
and e, (Table 6.13). 
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TABLE 6,9 (0-3) 


3.333 0.333 


Aol! 14.667 || 1.667 


| 


3.667 0.667 


| 4. 235 


Ais 


6 || —|| Lt |-H6.667 o.667 | | 0.667 


1.667 0,25 |—1 |—1.194 
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The columns (a; §)x and 6 are filled in the same way as under the first 
algorithm (see 5-5). The column 4@, is related to the vector aaa a 
like the vector A, to A, under the first algorithm: 

& =e, + X,A,—Y,As,, 
- Ci +X Xin for l#r, 
0 |e t+*Xre—y, for i=r. 


or 


In the upper part of the auxiliary tableau we introduce an (m+2)-th row 
(a; §) whose entries are the boundary values of the prcblem variables, 

When solving a problem givenincanonical form, ineachiteration three rows 
(A, X” and 6) are added in the lower part of the auxiliary tableau (Table 
6.14). In a problem with bilateral restraints in each iteration, moreover, 
it is necessary to fill the row X” containing the modifiers a, f of the extra- 
basis variables of the current pseudoprogram., 

The rows X! and A of the auxiliary tableau are computed following the 
general rules stated in 6-1 and 6-2 in connection with the second algorithm. 
The row 9 is computed precisely as under the first algorithm for a prob- 
lem with bilateral restraints (see 5-5). 

Let us now briefly outline the sequence of computations in a single 
(¢+ 1)-th iteration. 

At the beginning of the (/+1)-th iteration we assume to have at our dis- 
posal the principal /-th tableau (except the columns é@, and A,) and the rows 
of the auxiliary tableau corresponding to the /-th iteration. 

Pseudoprogram X is a solution of the problem if all the positions in 
column 6 of the principal /-th tableau are crossed out (case (a)). If the 
pseudoprogram is not a feasible program of the probiem, we proceed with 
the solution process, The vector A, with the largest deviation 8" is eli- 
minated from the basis. The rows X”, A“ and xX of the auxiliary tableau 
are then filled. The row X is filled following the same rules as in the 
first algorithm (see 5-5), The parameters Aj’ are computed from recur- 
rence formulas (6.7) using the elements of the preceding rows (A¢-", X@7)) 
0’-"), The components of row X are computed, as in a problem in cano- 
nical form, from (6.5). 

Rows X and xX; enable us to perform a limited unsolvability test. (The 
complete test at this stage necessitates the computation of all x‘ for which 
x,, <0.) Inconsistency of problem restraints is established in one of the fol- 
lowing cases: : 

(i) 8” with a modifier ag, x,,20 for x,=a, and x,,<0 for x,=B6,(/€l)); 

(ii) 6 with a modifier 8, x,,<0 for x,=a,and x,, 20 for x,=,(/€ /y). 

It is clear that unsolvability, if the problem is actually unsolvable, is 
established under the second algorithm at a later stage than under the first. 
The tests for case (a) and (b) complete the first stage of the iteration. 

We proceed with the second stage if case (c) obtains. In the second stage 
we choose the vector A, to be introduced into the basis and fill the principal 
(i+1)-th tableau (except for columns é and A,). 

The computations are made in the following order. The row 6” of the 

auxiliary tableau is filled according to the rules discussed in connection 
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with the first algorithm (see 5-5). The vector A, on which 


_ d 
8) = min 60 


is obtained is introduced into the basis, In the degenerate case, as long as 
no cycling has been detected, the simplified rule for the choice of A, (from 
the lowest or the highest index) should be applied. The exact rule guaran- 
teeing against cycling has been stated in 4-4. Observe that this rule re- 
quires knowing the coefficients x{? in the expansion of the vectors A, in 


terms of the basis of pseudoprogram X“, 


no } 
| Establish 8,>0 | Case (a) 
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yes | 
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The parameters x;, appear in the first-algorithm tableaus. In the second 
algorithm we compute (from (6.2)) only the coefficients x{f— the elements 
of the last column 4fof the principal /-th tableau; and x!)—the elements 
of the row X* of the auxiliary tableau. When the vector A, to be introduced 
into the basis is uniquely chosen following the exact rule, the coefficients 
x{? must be computed in the degenerate case whenever neccesary from 
formulas (6.2). The application of the rule guaranteeing against cycling in 
the second algorithm involves considerably more complicated computations 
than in the first. 


TABLE 6,16 


Having determined the elements x2 of the last column in the principal 


i-th tableau, we proceed to fill the main part of the principal (/+1)-th 
tableau. The columns e,(j=1, 2, ..., m)of the (/+1)}th tableau are expressed 
in terms of the elements of the [-th tableau using recurzence formulas 
(6.3). The component of the successive pseudoprogram, i.e,, the entries 
of column e, in the (/+!)-th tableau are determined using the same re- 
currence formulas from the entries in column @, of the /-th tableau: 
() 
a) —* 6 for t#r, 
*rk 
ey = 3(t) : 
— for i=r. 
al 
The columns (a; f)y and 4 are transformed as in the first algorithm. The 
value of the linear form is transformed between two successive iterations 
according to formula (5.17). 
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The (m+1)-th row of the principal tableau is required only to compute 
the initial values of the parameters A, In subsequent stages the kernel 
components of the successive support program of the dual problem are re- 
quired for control purposes only. The components y,=e,,,,,;,Y=1.2. ...,m) 
can be computed either directly from (6.1), or from recurrence formulas 
(6.3). Another possibility is to determine the parameters A, simultaneously 
from recurrence formulas (6.7) and from the defining relationships (6.4). 

This sequence of computations produces after a finite number of iterations 
either case (a) or case (b). 

In Figure 6.8 we show a block diagram of the solution of a linear-pro- 
gramming problem with bilateral restraints according to the second dual 
algorithm, 

In Table 6.15 (principal tableaus 0-2) and Table 6.16 (auxiliary tableau) 
we give the process of solution according tothe second dual algorithm of 
the linear-programming problem with bilateral restraints considered in 
5-5 in connection with the first algorithm. 


§ 7. Methods of determining the initial support 
program of the dual problem 


7-1. The solution of a linear-programming problem by the dual method 
sets out from a known dual basis, 

In some cases the concrete form of the restraints or the physical mean- 
ing of the problem makes it possible to establish an initial support program 
of the dual problem, and consequently the dual basis, easily. Among such 
problems, as we have seen in 3-7, are linear-programming problems with 
bilateral restraints. In 7-2 we shall consider another class of problems 
where an initial dual basis can be found directly. 

In other cases we can easily determine an initial feasible program of the 
dual problem, without being sure that this is a support program. Examples 
of such problems are given in 7-2. We should, naturally, expect that the 
construction of an initial support program is facilitated if a feasible pro- 
gram of the problem is known. In 7-3 we describe a method; and in 7-4 
an algorithm for constructing a support program from any feasible program. 
In 7-5 and 7-6 the method described is illustrated by examples. 

In 7-7 and 7-8 we consider two ways for solving a problem by the dual 
method in cases when the determination of an initial, even feasible, pro- 
gram involves some difficulties. 

7-2, Consider a class of linear-programming problems where the initial 
dual basis can be determined without computation, 

Let the linear-programming problem be given in the following form: 

Maximize the linear form 


L(X)= Qeyr, (7.1) 

subject to the conditions 
p? a,,x,=6,  {t=1,2, ..., m, (7.2) 
x, 20, Yee ar eee (7.3) 
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Now let all the coefficients c,(/=1, 2,..., a) 0f linear form (7.1) be non- 
positive. 

The problem reduces to canonical form when restraints (7.2), (7.3) are 
replaced by 


7.4 
Bi 84k) —¥ y= By i=1, 2,..., m, ( ‘ ) 


x,220, fs=1,2,..., atm. (7.5) 


The dual problem of problem (7.1), (7.4), (7.5) is stated as follows: 
Minimize the linear form 


L(Y= hoy, a 
subject to the conditions 7 
m 
BV Cp J=1, Sere M1, (7.7) 
—y, 20, i=1, Zee Me (7.8) 
The vector Y= (0, eee 0) 
~n ae 


is obviously a support program of the dual problem (7.6)—(7.8). If all c,<0, 
the support program Y is also nondegenerate, The dual basis thus com- 
prises the vectors 


a an. 
Anei = (0,...,0, —1,0,..., 0). 
—_—_——— ee 
mm" 
The vectors A,,,(/=1, 2,..., m)thus constitute the basis of the initial pseudo- 


program, The basis components of the initial pseudoprogram are equal to 
minus the corresponding components of the constraint vector. Similarly, 
the coefficients x,, in the expansion of the restraint vectors A, in terms of 
the initial pseudoprogram basis are equal to minus the corresponding com- 
ponents of these vectors, 

Now consider a class of problems where the initial support program 
cannot be indicated that easily, yet a feasible program can be established 
without difficulty. 

Let in problem (1,1)—(1.3) a certain fixed component a,, of all the re- 
straint vectors A, be positive. Consider the vector 

d 


—o—_—e— 
Y= (0, coos 0, Mt» 0, oeey 0), 
Ry a 
where si 
max ZL, 
4 eee 


The vector Y satisfies all the restraints of problem (1.4)—(1.5) and is, 
therefore, a feasible program of the dual problem, Vector Y, unfortunately, 
is generally not a support program, Itis, therefore, interesting to study 
the method enabling us to obtain a support program from a known feasible 
program. 

7-3, The method of determining a support program discussed in the 
following is called the gradient method. We first consider, in geo- 
metrical concepts, the essentials of this method. 
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The system of inequality restraints of the dual problem defines in the 
m-dimensional space of the variables y,, y,,..., ¥, a polyhedral set S, the set 
of feasible programs of the dual problem, Each support program corres- 
ponds to a vertex of set S, 

Let some feasible program Y, which does not coincide with any of the 
vertices of S, be given. The point yY’ lies inside some face S, of the poly- 
hedral set S (S,may coincide with S), The linear form of the dual prob- 
lem generates on S, some linear function, The gradient of this function is 
parallel to the projection of the constraint vector 8 on the face S,. We now 
let the point Y’ move in the direction of fastest decrease of the linear func- 
tion, until it meets the boundary S’. The new program Y’ lies inside the 
face S,, whose dimensionality is strictly less than that of S, Projecting 
the direction vector of the linear-form hyperplane 1 onto the face S,, we 
obtain the direction of fastest decrease of the linear function defined by L 
on S,. 

Proceeding in this way and lowering in each step the dimensionality of 
the corresponding faces, we finally arrive at a zero-dimensional face, i.e., 
a vertex of the polyhedral set S, This vertex corresponds to a support 
program of the dual problem. This support program is, as a rule, a fairly 
satisfactory approximation to the solution of the problem, This is quite 
obvious: we moved from Y’to Y” andfrom /” to Y’’’ etc,, in the direction 
of fastest decrease of the linear form ZL on the corresponding faces of set S. 

The gradient method may lead to a face S, parallel to the linear-form 
hyperplane. The linear form retains a constant value on this face. Further 
motion is possible, generally speaking, in any direction, as long as it leads 
to the boundary of this face, If the face S, is a polyhedron, rectilinear 
motion in any direction will lead to a program image on a face of lower di- 
mensionality, The choice of the direction of motion of the point Y”) is some- 
what limited, if S, is an unbounded polyhedral set, Itis, nevertheless, 
possible to reach the boundary of the face in question. 

We will now interpret these geometrical considerations analytically. 

Let y’=(y',..., y,,) be a feasible program of the dual problem, Without 
loss of generality, we may take 


m 
PETE. > Cp J=1, 2, wees Ris (7.9) 
Deu = ep j=n, t+, @eey A. (7.10) 


In particular, a, may be equal to a, 
Now, among equalities (7.10) let there be r linearly independent equa- 
tions. If r=m, Y’is a support program of the problem, Let r<m. 
Consider the system of equalities and inequalities specified by program 
y: 


AUN elp JN rey My (7.11) 
p> ii = Cy J=ntl,..., Am (7.12) 


Relationships (7.11), (7.12) define a face S, of the polyhedral restraint 
set S of the dual problem; the set S,is the face with minimum dimensiona- 
lity which still contains the point Y/Y” 

We express r variables (to be specific, y,, y,, ..., y,) from (7,12) in terms 
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of the remaining m-r variables 


Y= By Aiyit dp f=1, 2, sway Ts (7.13) 
We now insert the expressions for y,(i=1, 2, ..., 7) from (7,13) into (7.11) and 
obtain a system of inequalities for Y,.,, Yess cor Ve! 

Le! 

Pe a's yizec, JH, 2, ..., a. (7.14) 
System (7.14) obviously defines the necessary and sufficient conditions for 
the vector Y=(y,, .-.; Ym), Whose components satisfy equalities (7.12) (or 
(7.13)), to be a feasible program of problem (1.4)—(1.5). 

We now insert the expressions for y,, y,, ..., y, from (7.13) into (1.4) and 


obtain a linear function of the variables y,,,,..., y,: 


eS m 
L’'= Db by,+4,. (7.15) 
l=zr+i 
Let us now move the point y’= (yi, ..., y,,)in the direction of fastest de- 


crease of the function (7.15) until at least one of the inequalities (7.14) re- 
duces to an equality. The direction of fastest decrease of L’ is defined by 
minus its gradient vector 


— B’ =—(b' b’). 


r+yo oo °t mm 


It is assumed that linear form (7.15) does not retain a constant value on 
face S.. In other words, some of the coefficients 6, i=r+1,...,m are non- 
zero. Let the new program be denoted by Y” The point Y” lies in a face 
of lower dimensionality than the dimensionality of the face in which Y’ lies, 

Analytically, motion in the direction of fastest decrease indicates trans- 
formation from Y’ to Y’(§), where 


y, )=y,—0b, tor+1, r+2,..., m, 6>0, 


The remaining components of Y’ (6) are determined from (7.12) or (7.13). 
Let us now compute the parameters A,(6) for Y’ (6): 


A, O= 3) 04% 0)—ey= S 91,0,—06)—¢, = 
=Ay—0 3 910, (71% sa, 
Let 
(7.16) 
whence 
A, (0)= A,—6p, (7.17) 


The point Y’ (6) belongs to the polyhedral set defined by conditions (7.11), 
(7.12), i.e., the face S, of the set S, for all 6 satisfying conditions of the 
form (7.14), or, equivalently, 


A, (8) =0, J=1, 2, ..., a. 
From (7.17), we obtain the domain of definition of 6: 

A,2>, /=1, 2,..., 4. (7.18) 
According to (7.9), A,>0 for j=1, 2, ..., a, Therefore, if all p,<0, 
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we may take 0 arbitrarily large and Y’ (6) will not be outside the domain of 


definition of Z’. This indicates that L’forp,<0 (j=1, 2, ..., a,) is unbounded 
below in the set of the variables y,,, ..., y, Satisfying (7.14). In other 
words, for p,<0(/=1, 2, ..., 2,) the dual problem is unsolvable. 
Now let some of the p, be positive. Taking 
6. = min “4, (7.19) 
ni>o Hy 
we obtain program Y’=(y’, y’,...,9°,), where y=y)(0,) for i=r+l, ..., m, 
and the components y’, ..., y; are expressed from (7.13) in terms of the 


other known variables. 
From the definition of Y~% 


m 


Bs tM = Op jJ=a +1, ..., 4. 


Let 6, be obtained on j=, 


.= min a ie ‘ 

py>eoBy Hr 
Hence, A, (6,)=0, or, equivalently, 
m 


Dy Oipy, = Cy (7.20) 
The equality 


m 


ry ZinVi= Cz (7.21) 


is linearly independent of system (7.12). Indeed, system (7.12) is equivalent 
to (7.13). Hence, (7.21) can be written as 


m 

ere Fi = Cp: (7.22) 
From (7.16), p,>0 implies that at least one of the a, is nonzero, Let 

a’, #0 and the matrix of coefficients D of conditions (7.13), (7.22) have the 

form 


10...0 —4@,4,,1... —d,, 
O01...0 —d,4,,.... —d,, 
rk. de 28 Te Se wa, Se 
0:0 = das —d,, 
00 -O@ . a 


The determinant of the submatrix comprising the first r columns of the 
matrix D and the column containing the element ai, is equal to a’, and is 
thus nonzero. This indicates that (7.22) is linearly independent of the sys- 
tem (7.12). 

Program Y’ thus reduces at least r+1 linearly independent inequalities 


from among the restraints of the dual problem to equalities. Here 
~ m m 
L(Y = 2 b= DOM (6,) +0) = 


m m m m 
= by, +b’ —6 b‘)* ’y’ 4 b= y= L(y’ 
> 49, + 0 0 2, CM < Db +K= Dd = LY). 


s=r+1 
Thus, if at least one of the coefficients b) (i=r+1, ...,m)is nonzero, each 
step of the gradient method decreases the linear form of the dual problem. 
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Now let all 6 be zero. Geometrically, this indicates that the face $, 
containing the program Y’ of the problem, is parallel to the linear-form 
hyperplane and, consequently, L' retains a constant value (equal to 6) on 
S,. In this case the boundary of S, can be attained if Y’is moved as follows 


Year O=¥,4, A, 
Y=» bar4+2, ..., m. 


(Any of the y' with i>rcan, obviously, play the role of the variable y,,,.) 
The conditions for the domain of definition of 6 are 


(7,23) 


A, (t)= > a; (8) —c) 0, jm l, 2, 0.05 My 


f=r+i 


or, equivalently, 


A , (6) = A,—fe’ ;= 9, J= l, 2, sets Ry. (7.24) 
All a’ 


ra1, ; cannot vanish simultaneously. Otherwise, no m—r linearly in- 
dependent inequalities could be chosen from (7,11) and the system of re- 
straints (1.5) would contain less than m independent relationships. 

The parameter 6, i.e., the limiting value of 6 for which we remain in 
the face S, is determined, according to (7.24), as follows: 


min —“4-, if some @,4s,7 are positive 
Fr 41,/7° ated 


0 (7.25) 
° — min (——“L_} » if ay ;=Oforj= 
Ora <? Frei. f Se 2h. esc Mis 
Observe that for a),, <0 (/=1, 2, ...,a,) 9% is negative. The number of 


linearly independent restraints reduced by Y’=Y’ (§,) to equalities increases. 
The linear form, however, retains its previous value, Proceeding with 
this technique we, necessarily, obtain a support program of the dual prob- 
lem having the same value of the linear form. This, obviously, requires 
at most m steps. 

The gradient method for constructing a support program from a given 
feasible program was discussed here with reference to the dual problem, 
because it is generally easier to determine a feasible program of the dual 
problem (1,.4)—(1.5) than a feasible program of the primal problem (1.1)- 
(1.3). If, however, a feasible program of the primal problem is known, 
the same gradient method will produce a support program of the primal 
problem. Only in this case it is advisable to modify the computational pro- 
cedure so that it allows for the specific features of linear-programming 
problems in canonical form. 

7-4. Computations by the gradient method can be made quite compact. 
The transition from one face of the polyhedral restraint set to another face 
of lower dimensionality involves solving a system of linear equations by 
the method of successive elimination of variables. The computational pro- 
cedure of the gradient method can be represented ina succession of tableaus, 
The initial zeroeth tableau is shown in Table 6.17, Here 

b,=a;,, i= 1, 2, ..., mM, 
C;= 4m, ]> jJ=1, 2, coe, AR, 
am+i,0 =ba5, =¢(= L (X). 


In the columns of the tableau (in the first m+1 positions) we have the 
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components of the augmented constraint vector A, and the augmented re- 
straint vectors A;_ 
A,= B= (5,, ere eg b,, L (X)) =(2,,, oeeys aay) Qm+1, o)s 


A,= (4, ) vise Aggy C)) = (4, ) coer Bay Om+s, /); 
J 162, Saeed 
We shall call the matrix fa,,J (i=1, 2, ..., m, m+1; /=0, 1, 2,..., a) the main part 
of the tableau. 
TABLE 6.17 


Zeroeth tableau 


In the last (m+ 2)-th row of the tableau we write the differences (A) between 
the left- and the right-hand sides of restraints (1.5) obtained when some 
feasible program Y’of the problem is inserted for Y. All the elements in 
this row are nonnegative. 

The position A, is left empty. 

Let the elements of the (m+ 2)-th row A, eon in, vanish for Y=Y' Pro- 
gram Y’ thus reduces a—a, restraints in (1.5) to equalities. We apply these 
equations to eliminate part of the variables from the other restraints in 
(1.5) and from the linear form, It is convenient to eliminate the variables 
successively, one by one, transforming the system using the recurrence 
formulas of the Gauss complete reduction method. 

We now use the ¢-th equation to express one of the variables, say y, 
(the only requirement imposed on the index k<mis that ay;, #0) and insert 
the result into the other restraints and into the linear form. The main 
part of the zeroeth tableau (the matrix |ja,,j) is thus transformed into the 
main part of the lst tableau (the matrix ja,,j). The main parts of two suc- 
cessive tableaus are related by the recurrence formulas 


seas ay, for j#t,, 


‘= 7.26 
“i Ait for j=? 
By, . 
i=1, 2,..., m, m+; j=0, 1, 2, ..., 24 
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or, equivalently, 
; @,-— O44, ‘i ay, for J a t 


a “ for j=t, (7.26') 
kt, 


i=1, 2, ..., m, m+1; /=0, 1, 2, ..., a. 


We see from formulas (7.26) that the k-th row of the Ist tableau reduces to 
the unit row vector 

+ _f90 for j#¥#,, 

a= { 1 for j=t?,. 


We shall call these unit row vectors transformed rows. 

If n—n,> 1, the (m+2)-th row of the zeroeth tableau is not transformed 
(A’ == A). 

In the same way we pass from the Ist tableau to the 2nd tableau, etc., 
until all the n—nza, have been used to eliminate the variables and transform 
the matrix oy ll (@=1, ..., m+hio j=0, 1, .-., n).Consequently, we obtain a 
tableau with r transfornied rows, i.e., (a+ 1)-dimensional unit vectors. 
The unit elements of these rows are written in columns corresponding to 
zero A, The process is terminated when zero elements stand in all the 
intersections of the columns A,=0 and nontransformed rows, 

For n=n, there is no need for all these transformations. 

If r=m, the problem is solved, i.e., the vectors A, corresponding to 
the units of the transformed rows constitute the basis of a support program 
of the dual problem. 

Now let r<m. In allthetableaus, except the last, the (m+2}-th row is not 
transformed, A=A’=A’=... In the last tableau, the one with , transformed 
rows, we should transform the row A into the row A”=A(6,) according to 
(see (7.17) and (7.24)) 

2 
A? =A,(6,) = A,—6 Op), eet) 
where 


i 
(r) (7) 
Q4/4so, if at least one 
po = 2, ee (7.28) 
/ as) = is nonzero (i=r+1,...m), 


a,, 4, if all bs are zero (i=r+,..., m). 


Here s, are the indices of the nontransformed rows, j corresponds to 
columns with A,>0, 


A 
/ ° (r 
min a if some of the p"”? 


uy > 
gn are positive, 
— A; 7.29 
— min -—4 , if all b{ are ( ) 
wy<o\ By 


zero and all p? <0. 


In the expression for p» the subscript s,,, is the index of any of the non- 
transformed rows where all the elements corresponding to A,>0 are zero. 

In the tableaus corresponding to the steps of the gradient method in 
which A is transformed it is advisable to write the parameters A not in 
the (m+2)-th row (as in the zeroeth tableau), but in the (m+ 4)-th row. In 


these tableaus ,» are written in the (m+ 2)-th row, and 6”, i.e., the values 


A 
of the ratio ZL, in the (m+3)-th row. The elements of the row 6” are 
By 
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computed only for those positions where p>0. The parameters yp’), in 


turn, are computed only for A,>0. In cases when all bf are zero and all 
nu <0, entries should be made in the positions of the row 6” corresponding 
to negative p" 

The problem is unsolvable if some of the 6 
B,<O. 

If there is no indication of unsolvability of the problem, the gradient 
method should be resumed. The transformation of the row A will intro- 
duce one or several additional zeros into the row A” of the r-th tableau. 

The main part of the r-th tableau is again transformed using recurrence 
formulas (7.26) until the corresponding number of variables have been eli- 
minated from the restraints of the dual problem, Recurrence formula 
(7.27) is then applied to transform A. 

The process is continued until all the m first rows of the tableau are 
reduced to unit vectors or until unsolvability of the problem is established. 
The restraint vectors corresponding to units in the transformed rows con- 
stitute the required basis of the dual problem. 

Until now all the rows of the tableau were assumed linearly independent. 
We will now drop this assumption. 

Let the computations connected with one of the tableaus (say, the p-th 
tableau) produce a row (say the /-th row) whose nonzero elements are located 
only in the columns containing the units of the transformed rows. Then, 
one of the following two cases may arise, depending on whether the /-th 
element in column A, is zero or nonzero: 

(i) the rank of the restraint matrix |/a@;,{| (4=1, 2, 


less than m, if a)=0% —0; 


{ are nonzero and yet all 


.., mj=i1, 2, ..., ais 


(ii) the problem is unsolvable, if aly) = bp) # 0, 
Indeed, by assumption, the /-th row of the p-th tableau is a linear com- 
bination of the transformed rows: 


| a 
af} = 3 a,c, 7.30 
BE (7.30) 
(a,=0 for the indices corresponding to nontransformed rows). 
The equivalent dual problem obtained from the transformations which 
produced the p-th tableau is written in the following form: 
Minimize the linear form 


Li 
L®) (y). (p) 
(Y)= p> eT 
subject to the conditions 


Dolly ae. 


Following (7.30) we rewrite the restraints of the equivalent dual problem 
in the form 


> aif] (y+ 4,y,) > ey 


xm 
en Ot 


Let y’=(ji, +) Ji «+> Ym) be a feasible program of the dual problem, Let 


m=eytay for igel. 
The vector 
Y= (J, eees Ju eons Ya) 
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where 
yitay=yn inl, (7.31) 
is obviously a feasible program of the dual problem. Clearly, for any y, 


there exist y, for which (7.31) are satisfied. 
Let us now compute the value of the linear form Zon program Y: 


LO (Y= BoP yi—ayt+oP y= Bory: + y, (o? —> a?) (7.32) 
ted izt tz 


First, consider the case when 
m 
bf = Dad, 
tH! 


i.e., when bY) =a"? is the same linear combination of the corresponding 
parameters of the transformed rows as the other entries of the /-th row. 
In this case the entire /-th row of the p-th tableau is a linear combination 
of other rows, Transformation of the tableau does not change the rank of 
lla:,{| (é=1, 2, ..., m, m+1; 7=0, 1, ..., A). 

Thus, we reach the conclusion that some of the problem restraints are 
linearly dependent and the rank of the restraint matrix is thus less than m 
(case (i)). The /-th row of the p-th tableau is crossed out in this case and 
left out in future computations. 

Now let 57 + } a,b%. We have already seen that there exist feasible 

f=1fzi 
programs satisfying (7.31) with an arbitrary y,, Therefore, if the coeffi- 
cient of y, in (7.32) is nonzero, ZL” and, consequently, linear form L of the 
initial dual problem is unbounded in the set of feasible programs (case (ii)). 

One more remark on the algorithm of the gradient method. 

The coefficients c,=a,,,,, are required to compute A, in the zeroeth 
tableau (in subsequent tableaus, A, either does not change, or is trans- 
formed according to recurrence formulas) and to compute the components 
of the support program of the dual problem, It is often necessary, however, 
to determine not a support program of the dual problem, but rather the 
corresponding dual basis. In these cases the (m+ 1)-th row can be omitted 
from all tableaus, except the zeroeth tableau. 

7-5. We now illustrate the application of the gradient method by an ap- 
propriate example. 


Example 1. Minimize the linear form 
L(Y) = 2%, +34. +34 C82) 


subject to the conditions 
Wt+%,+ ysl, 
2y,+ %t+2ysS i, 
At Atty, =], 
2y, + 2y, + 3y3 > 2, 
2y, + 2y, + 24, > 1. 


(7.34) 


Solution. Here m=3, na=5. It can easily be verified that the vector y=(0, 0,1) is a feasible pro- 
gram of problem (7.33)-(7.34). 

The entire computational procedure by which the support program is determined is shown in Table 
6.18 (0-3). In the zeroeth tableau we write the problem restraints. Inthe (m+ 2)-th (fifth) row we indicate 
the values of 4, i.e., the differences between the left- and the right-hand sides of restraints (7.34) for 
Y= (0, 0,1). 
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TABLE 6.18 (0-3) 


No. of 
tableau 
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We see that the program Y reduces only the first restraint in (7.34) (’=1) to an equality. We eliminate 
one variable, say y,, from the system. The main part of the new tableau (1st tableau) is transformed ac- 
cording to recurrence formulas (7.26). The direction element of the transformation is a,,=1. We have, 
for example 


Og = Fyq— 2, Ogg = 3— 2-3 = —3. 


In our case r=], Therefore, in the first step, we transform the row A also. The parameters A’are com- 
puted from 


c t 4 
A; =A ,— Op > (7.35) 
In the (m-+2)-th (fifth) row of the 1-st tableau we give the values of By . The parameters yy, are com- 


puted as products of columns A, and A yj (the first m entries) in the Ist tableau, In taking the product, only 
the elements of nontransformed rows are taken into consideration. Thus, for example, 


Hy =2),0 55 -+05.0,,=0(—1)-+(— 3) (— 3) =9. 


, A 7 
In the (m-+3)-th row of the Ist tableau we write =o The least value 6, of the row 0 is _ 0.0769; 


it is obtained on Ay. Applying 0, and the parameters Ky we compute from (7.35) the elements A, of the 
(m +4) -th row (A’) in the Ist tableau, Now, single out the column A, with A,=0 and repeat the entire 
sequence of operations which leads to the 2nd tableau, 

In the 2nd tableau three columns are singled out. Starting with the last of these columns, we fill the 
last, 3rd, tableau. 

The components of the support program Y=(y,, Ys, ¥,3) are obtained from the (m+1)-th, 4th, row. 
To determine y; we move along the i-th row of the 3rd tableau to the position in which | is written. The 
corresponding entry in the (m-+1)-th, 4th, row is the component required. Thus, 


y=1, y,=0, y,=0. 


There is thus no need to fill the rows p’’’, 6°’, A’””, 
It can easily be verified that the support program Y=(I, 0, 0) solves problem (7.33)~(7.34). 


The first support program obtained need not be the optimal program of 
the problem in question. In general, when the support program obtained 
by the gradient method does not solve the problem, the dual method should 
be further applied. To do so, besides the support program, the corres- 
ponding matrix |le,,|| of the coefficients in the expansion of the m-dimen- 
sional vectors e,(//=1, 2, ..., m) in terms of the basis obtained is also re- 
quired. The elements e;, of this matrix satisfy the equations 


m 1 for i=jJ, 
Dene) ctor tas (7.36) 


Here /,=(s,, ..., S,)is the set of indices of the basis vectors. 
The system of equations 


Di aii=ep JE Fy» 


defining the components y; of the support program Y differs from system 
(7.36) only in the right-hand side, Itis, therefore, advisable to solve these 
systems simultaneously. If we know in advance which of the restraint 
vectors enter the required basis, we could have applied the gradient algo- 
rithm to obtain, together with the support program, the corresponding 
matrix |le;,|. Todo so we would have to write the unit row vectors corres- 
ponding to the basis vectors in the rows m-+-2, m+3, ..., 2m+1o0f the zeroeth 
tableau and successively transform them following the formulas used in 
transforming the main part of the gradient tableaus. Asa result, simul- 
taneously with the support program, we would have obtained the matrix ||e;,|| 
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at the intersection of the additional rows and the basis columns. Note that 
gradient transformations do not affect the unit vector e, until the compo- 
nents of the vector A, have been inserted into the system of equalities (7,12). 
Therefore, the additional unit row vectors can be introduced gradually, as 
the new basis is being formed. This enables us to combine the construc- 
tion of the matrix |le;,|| with the determination of the support program YF, 

We proceed from the last computational tableau of the gradient method 
(say the N-th tableau) in order to fill the main part of the principal zeroeth 
tableau when solving the problem under the second dual algorithm. 

The element e,, of the principal zeroeth tableau (i=1, 2, ..., m; j=l, 
2, ..., m)is written in the A-th tableau in the row corresponding to the vec- 
tor e,. Here s; is the index of the vector A, occupying the i-th basis po- 
sition. The components of the support program Y are written in the 
(m+ 1)-th row of the N-th tableau. 

To obtain e,, (the element in the j-th position in the i-th row of the prin- 
cipal zeroeth tableau) we must locate the unit in the j-th row (/=1, 2, ..., m) 
of the N-th tableau, The element of the row e, in the N-th tableau located 
in the column containing this unit is the required e,;,, The element in the 
(m-+1)-th row located inthe same column specifies the component y,=e,,,,, 
of the initial support program Y, The basis components of the initial 
pseudoprogram (elements of the column e, in the principal zeroeth tableau) 
are computed from the general formulas 


ms 
Xin = Dy eiabr. 
A=1 
We now illustrate the above. 
Example 2, Using the dual method, maximize the linear form 
L (X) =x, + 2x, +3x, +2, + 2x,+ 3x, +2, (7,37) 


subject to the conditions 


2x,+3x,+ Xyt+ % F 3x, +2x, +2e, =8, 

X¥, + 2x, +3x, + 2x, +0,5x,+ 2+ x, =6, 

2x, + Xygt-3x,+ XAyt Aye t3x,+ x,=7, 
xj=20, j=l, 2, ..., 7. 


X,+ xX,+2e,4+3x, b 2x,+3x,+ x,=7, 
(1, 38) 


Solution, The corresponding dual problem is stated as follows: 
Minimize the linear form 


L(Y) =7y, +8y2 +6y,+7He 
subject to the conditions 
Yi: + 24+ Ytaysl, 
Y:+3y,+ yet Yae?, 
2+ Yet Sy, +3y, 3, 
S¥4;+ Wt Wt wel, 
2y, + 3y, + 0.5y, + 2y, => 2, 
BY, + 2y,+ Ya t+ 3yy 3, 
wAt+BW+ Yt wel. 


All the components of the restraint vectors are positive. Therefore, following the suggestions of 7-2, 
we take as the initial feasible program any vector of the system 


Y,=(%, 0, 0, 0); Y,=(0, Ya, 0, 0); Y,=(0, 0, Ya, 0); 


Y,=(0, 0, 0, 4%), 
where 


Yy;== max ate, 
Is f/ar Fz 


Starting with the feasible program Y=Y,=3(0, 0, 4, 0), we compute the support program and the 
corresponding parameters e; by the gradient method, 
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The entire computational procedure is given in Table 6.19 (0-4), In the Ist tableau, the row A gives 
the differences between the left- and the right-hand sides of the restraints of the dual problem for the given 
feasible program: 

Aj= 4a,;—C;, j=l, 2, ewes 7. 


The row A has one zero corresponding to the vector A,, In the zeroeth tableau we should, therefore, add 

a row whose elements are the components of the unit vector e,=(0, 0, 0, 0, 1, 0, 0). The rows e, are best 
arranged directly below the rows in the main part of the tableau, so as to facilitate their transformation by 
means of the same recurrence formulas. 

The column 4, is taken as the direction column of the transformation. Any of the rows with a nonzero 
element in the position corresponding to column A, can be taken as the direction row. To be specific, we 
choose the first row. The direction element of the transformation a,= 2, The main part and the row é, 
of the zeroeth tableu are transformed according to recurrence formulas (7,26). The parameters p’, 6’, and 
A, are then computed and written in the corresponding rows of the lst tableau. Transformation to the 2nd 
tableau is carried out analogously. 

When filling the 3rd tableau we encounter the special case mentioned in 7-3: 


a,, =, =0, i=l, 2, 3, 4. 
According to the rule discussed in 7-4 (formula (7.28)), one of the nontransformed rows should be transferred 
to row ’”, In our case, there remains a single nontransformed row, namely the fourth, Therefore p, =a 
Some of the elements in row p’” are positive. In the positions of the row 6’ corresponding to p’”>0 
we write 


det 


aj 


/ Tre 
By; 


The positions of the row 6’ for which "<0 are crossed out. The parameter @)”’ is determined from 
(7.29), and the row A” is filled as usual, 

Formation of the 4th tableau completes the gradient method. The 4th tableau contains all the infor- 
mation needed on the required support program Y =(¥;, ...-, Ym) Of the dual problem and the corresponding 
matrix |l|e;;||, The basis of the program comprises the vectors A,, A,, As, and Ag. 

In Table 6,20 (principal tableaus 0—1) and in Table 6,21 (auxiliary tableau) we show the process of 
solution of problem (7,37)-(7,38) according to the second dual algorithm. The principal zeroeth tableau has 
been filled according to the data obtained when computing the support program Y by the gradient method 
(see the last 4th tableau of Table 6,19), Thus, for instance, ¢@,,, corresponding to the restraint vector 
A,=A,, is taken from the 4th tableau (Table 6.19) from the row e, =e,. In the row j=2 of the 4th 
tableau the unit is in column A,. The parameter e, is at the intersection of row é, and column 
Ay: €y,=0.526. The element e,=2 is located in the 4th tableau (Table 6.19) at the intersection of row 
és, =e, and column A,;, which contains the only unit of the row j=1. 

The components of the initial support program Y, the elements in the last row of the principal zeroeth 
tableau, are determined in the same manner, In this case the (m-+1)-th row of the 4th tableau replaces 
the rows ey, Thus, for example, Y,=€y41,1==1, Ye=Cm41,2= 0.0526 , etc. 

The basis components x; of the initial pseudoprogram, i.e. , the elements in column e, of the initial 
zeroeth tableau, are computed as products of the elements e, of the i-th row of the tableau and the cor- 
responding components of the constraint vector. Thus, e.g. , 


Xjg=— 2-7 +0,526-8—0.737-6 + 1.895-7 = —0.947. 


The problem is then solved by the general rules of the second dual algorithm, 
The optimal program of the primal problem is 


X*=(0, 1.562, 0.375, 0.188, 0; 1.375, 0). 
The solution of the dual problem is 
Y* =(— 0.146, 0.354, 0.104, 0.875). 
The optimal value of the linear form is 
L(X*)=L (Y¥ *) =8.562. 


One iteration of the dual method was required in this case to solve the problem. This is due to the 
fact that the gradient method yielded a good first approximation, 

As we have already observed, this is not a chance phenomenon: the gradient method generally produces 
a support program which is close to the optimum, 
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7-6, When determining the initial support program by the gradient 
method we assumed that some feasible program of the problem was known, 
We mentioned before that for a wide class of problems the computation of 
a feasible program involves no difficulties. There are, nevertheless, 
problems in which the operations needed to find an initial feasible program 
are no less tedious than those necessary to compute a support program 
from a feasible program. In these cases it is advisable to follow the method 
discussed below. 

To restraints (1.2) of the linear-programming we add the condition 


X,+4%,4...+4+4%, = M, 
X,+%,4+...+4%,=M, x, = 0. 


or 


The new problem (in variables x,, x,, ..., x,) is called the augmented 
problem, 
The dual problem involves minimization of the linear form 


My, + D3 UAT (7.39) 
subject to the conditions r 
y+ 2D AY, Cys jJ=1, 2, ..., 2, 
Y, = 9. 


This is the augmented dual problem, 
The vector 


Y=(y,, 0, ..., 9), 
where 


Y.= max {c,, O}. 
/ 


can, obviously, be taken as a feasible program of the augmented dual prob- 
lem. 

Starting with program Y, we may apply the gradient method to derive a 
support program of the augmented dual problem, The resulting dual basis 
will be taken as the basis of the initial pseudoprogram and the augmented 
problem is then solved by the dual method. In the process of solution, M 
is assumed fairly large (larger than any number with which it is to be com- 
pared in the process). 

Let X*= (x, x, ..., x,) and Y*=(y,, yi, ..., ym) be the optimal programs of 
the primal and the dual augmented problems. Two cases may arise 


y.=0 or y>0. 


In the first case Y*=(y', ..., y,,) is obviously a solution of the dual prob- 
lem (1,4)-(1.5). Therefore X* =(x;, ..., x,) is an optimal program of problem 
(1.1)—(1.3): 


n ™ 
p>) jy + Ox = 2 iyi + M. 0, 


In the second case (y,>0), the set of feasible programs of the dual prob- 
lem is empty. Indeed, let Y=(y,, ..., y,) be a feasible program of problem 
(1.4)-(1.5). Then Y=(0, y,, ..., y,) is a feasible program of the augmented 
problem. For sufficiently large M, the value of the linear form (7.39) at 
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the point y*=(y%, y*, ..., y’%) will be higher than its value at the point Y= 
=(0, y,, ..+. Yq) and this contradicts the optimality of program Y*, Thus, 
for y’>0, problem (1.4)-—(1.5), and, therefore, its dual problem (1.1)—(1.3) 
are unsolvable. 

If when solving the augmented problem we obtain case (b), indicating 
inconsistency of restraints, this is a hint that the restraints of the primal 
problem are also inconsistent. Indeed, if X=(x,, ..., x,) is a feasible 


program of the primal problem then for Me dx, the vector X is a feasible 
=t 


program of the augmented problem also. 

Thus, this method enables us to find an optimal program of a solvable 
problem without first determining a feasible program of the dual problem. 

7-7. In conclusion we give another technique for solving linear-program- 
ming problems by the dual method, without preliminary tedious computa- 
tions of an initial dual basis. The technique is based on the proposition 
(3-7) that in a linear-programming problem with bilateral restraints any 
system of m linearly independent restraint vectors can be taken as the 
basis of an initial pseudoprogram. 

We take M (a number larger than any comparand to be used in the pro- 
cess of solution) as the upper bound of some variables in problem (1.1)— 
(1.3). The variables to be bounded above will be specified in what follows. 
The new problem will be called the bounded problem, 

Physical meaning or some features of the matrix of coefficients may 
enable us to isolate m linearly independent restraint vectors, This system 
can then be considered as the basis of some pseudoprogram of the problem, 

In the general case, when the features of the problem do not hint ata 
linearly independent system of vectors, the dual basis can be determined 
by successive transformation of equalities (1.2), following the complete 
reduction method. The components of the restraint vectors are transformed 
according to recurrence formulas of the form (5.16): 


aff =~ yy ters fol, 2, ..0) m 


As the direction element in each successive step we take any nonzero 
element which does not belong to the direction rows or columns of the pre- 
ceding steps. It may, however, turn out that at a certain step this rule 
leaves us with no direction elements. 

The direction rows and columns of the preceding steps are crossed out 
in the matrix. The resulting array is called a reduced matrix. The 
transformation of the restraint matrix and of the constraint vector should 
be discontinued when the reduced matrix becomes empty or a null matrix. 
The reduced matrix may become empty at some step if restraints (1.2) are 
consistent and linearly independent, In this case, the restraint vectors 
transformed in the preceding steps into unit vectors can be taken as the 
basis of the initial pseudoprogram, A reduced matrix is null in one of the 
two cases: 

(i) the problem restraints are inconsistent; in this case at least one of 
the components #{/ of the transformed restraint vector corresponding to 
a row of the reduced matrix is nonzero; 
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(ii) the problem restraints corresponding to the rows of the reduced 
matrix are linearly dependent on the other restraints; in this case b\ 50 
for all rows of the reduced matrix; the rows isolated in this way can be 
crossed out; the rank of the restraint matrix is now lower. 

These transformations of the matrix |ja,{| will, thus, eventually establish 
the unsolvability of the problem or produce a system of linearly independent 
restraint vectors, i.e., the basis of the initial pseudoprogram, In the 
course of transformation, we also obtain the coefficients x!) in the expan- 
sion of the restraint vectors in terms of the vectors of the initial basis. 
These parameters are sufficient for solving the problem by the first dual 
algorithm. 

The second algorithm also requires the inverse matrix 


-1 
Nes Mew = Uk@easllm 


(8) Sg .+-) Sq)=/,, The matrix |le,/|, can be obtained together with the deter- 
mination of the basis vectors if, from the start, we introduce to the right 
of the restraint matrix m unit (m-dimensional) vectors e, and transform 
the augmented matrix according to the above rules, 

A sequence of transformations will thus produce the complete system of 
linearly independent restraint vectors A,,...4,, and the corresponding 
matrix |lxijllen (OF lel, Of the expansion coefficients of the vectors A, (or 
e,) in the linearly independent system chosen. We then compute the para- 
meters 


liz 
A,= ms © Fig Ep 


The variables x, whose A, are negative are bounded above by a suffi- 
ciently large number M. We have thus obtained the bounded problem. 
The system of restraint vectors (A,;, ..., As) is the basis of a pseudoprogram 
of this problem. 

We should emphasize that in this bounded problem the variables with 
A,=0 (the basis components of the pseudoprogram included) are restrained 
only by the requirement of nonnegativity, whereas the remaining variables 
with negative A, are bilaterally restrained. 

The bounded problem is solved by the dual method, The remarks in 
5-5 and 6-5 concerning problems with bilateral restraints are taken into 
consideration. In the process of solution the number M is taken larger than 
any comparand which need arise, 

It follows from the preceding that in a finite number of steps we obtain 
case (a) or case (b). 

In case (a), the optimal value of the linear form of the bounded problem, 
as in any problem with bilateral restraints, is written according to (3.11) 
in the form 


L(X)= Zon AW; 


Here y,; are the components of the kernel Y of the solution of the problem 
dual with respect to the bounded problem, /, is the set of indices of the 
basis vectors of the vectors of the optimal program X,y, is the value of the 
extrabasis components of the solution. In our case 


M if J€2, 
Y=) 0 if J€2, 
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where £ is the set of indices of the extrabasis variables of the solution of 
the bounded problem equal to the upper bound M, 

Thus, in case (a) the maximum value of the linear form of the bounded 
problem can always be written as 


L (X) =L, (X) + ML, (X) =L, (X)—M J Ay. (7.40) 
Here 


L, (X= —¥ A, 0. (7.41) 


The basis components x, of the solution X of the bounded problem are 
the coefficients in the expansion of the vector B— SAW in terms of the basis 
/ 


vectors of the optimal program X, 


Y Y 
Xj, = 0 = Ba Yp 
lely 
or, equivalently, 


xi. =O —MD> xf). (7.42) 
j€B 


In case (a) two outcomes are possible: 
(i) L,(X)=0, (ii)L, (X)#0. 


In (i) the linear form is independent of M. The primal problem is 
solvable and its optimal program can easily be determined. Case (a) arises 
when the set 2 is empty or when 74 =0, If the set £ is empty, all the 

E 


extrabasis variables of program xX are zero and the solution of the primal 
problem coincides with the optimal program X of the bounded problem. 


The basis components of the solution x,,=0{" (see (7.42)) and are inde- 
pendent of M. If ps A,=0, then from (7.41) 4,=0 for jEE (the optimal 
€E 


program of the dual problem is degenerate). For the other (zero) extra- 


basis variables 
A,= 0. 


According to the optimality criterion (see Chapter 3, Theorem 5.2) a 
feasible program satisfying these conditions is an optimal program of the 
primal problem, though not necessarily a support program. As the para- 
meter M entering (7.42), we may take any M,>0 for which 


x,,= OM, Dd xf) SO. 


A support program can be obtained from the feasible program (if at all 
necessary) without difficulty, The computations are carried out according 
to the gradient method. 

In case (ii), when 


L,(x)= — By As>0 


the primal problem is unsolvable, since the linear form is unbounded in 
the set of feasible programs. 
In case (b), when the bounded problem is unsolvable, the primal problem 
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is not solvable either. The unsolvability of the problem can be either due 
to inconsistency of restraints or to the fact that the linear form is unbounded 
above. 

A rigorous theory of this method is based on the following two proposi- 
tions: 

1. There exists a number M,<oo such that for all M>M, the linear form 
of problem (1.1)—(1.3) is unbounded above in the set of its feasible pro- 


grams when oye 


2. There exists a number M,<oo such that for all M>M, unsolvability of 
the bounded problem implies unsolvability of the primal problem (1,1)—(1.3). 
The proof of these propositions is analogous to the proof of the corres- 
ponding theorems in the M-method. The reader, following the technique 
of Chapter 4, §7, will easily establish the validity of these propositions 

(see Exercise 9). 


We illustrate the preceding by an appropriate example. 
Example, Maximize the linear form 


L (X) = 10x, + 2x4 +43 +9x,+2%5+4%,+%, 
subject to the conditions 
3x, + 7x, +5x, +44, +425+ x, =, 
7x, + 15x, + 9x, + 9x, +2, + 2x, +x,=8, 
6x, + Llx, + 7x,4+6x,+2,+ x, +%,=6, 
xjor0, j=l, 2, ..., 7. 


Solution. We must first obtain a system of linearly independent vectors, In Table 6,22 (0-3) we 
give the sequence of transformations leading to the linearly independent vectors A,, Ag, and A, and the 
coefficients in the expansion of all the restraint vectors in terms of these linearly independent vectors, The 
primal problem is then (after the 3rd step) replaced by the following problem. 


TABLE 6.22 (0-3) 


Maximize the linear form 


L (X) = 10x, + 2x, +2, +92, + 2x5 +4, +2, (7.43) 
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subject to the conditions 


2x, +3x,+3%,-+ %44+%, =3, 
x, +4%,+2x, +3x, +%, =2, \ (7. 44) 
3x, + 4x, + 2x, + 2x, +x,=1, 

xj0, j=l, 2, ..., 7. (7. 45) 


The presence of unit restraint vectors in problem (7,43)-(7,45) and of positive right-hand sides in system 
(7.44) indicates that it is advisable to apply the simplex method, Nevertheless, for the sake of illustration, 
we employ the first dual algorithm, 

We compute the parameters A, corresponding to the restraint vectors, We have 


A,=—2, A,=12, A,=9, A=—2. 


Following the suggestions of the present article, we take M as the upper bound of the variables with 
negative A; The bounded problem corresponding to problem (7.43)—(7,45) requires maximization of linear 
form (7.43) subject to conditions (7.44), (7.45) and the additional restraints 


x,=M, xM. (7, 46) 


In Table 6,23 (0-3) we give the process of solution of the bounded problem by the first dual algorithm, 
The computations are carried out as suggested in 5-5, where the first algorithm is applied to problems with 
bilateral restraints, 

We see from the tableaus that after the first iteration M is omitted from the expressions for pseudo- 
program components and linear-form coefficients, The computations in subsequent iterations can be carried 
out effectively according to the first algorithm for the problem in canonical form, There is, thus, no need 
to fill the row X and the columns A,, 6, and (a; B)y. 

After three iterations, we obtain the solution 


X=(0, 0, 0, 1/2, 5/2, 1/2, 0). 


§ 8. The simplex method and the dual method 


8-1. In the present chapter the dual method was presented independently 
of the simplex method. We shall now show that the dual method is actually 
a variation of the simplex method applied to the solution of the dual problem, 

We rewrite the dual problem (1.4)—(1.5) in the following form: 

Minimize the linear form 


subject to the conditions a 
p> 2iJi— Yar j= ep (8.2) 
Yn+;= 9, 
j=1, 2, oeey Rn. (8.3) 


Consider the vectors 
i 
AM = (Bi, Digs coy a,,); 


/ 
Ow 


60s hag 0). 150; gay "0)?, 
nan 


C= (65. Ch heey C5)", 


and rewrite restraints (8.2) in the vector form 


m 


2 A° YB easrs= 6 (8.4) 


é=1 
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The primal problem (1.1)—(1.3) will be called problem (A) and the dual 
problem written in the form (8.1)—(8.3) problem (A), We shall solve prob- 
lem (A) by the simplex method. 

Problem (A) has m+n variables y,, y,, ...; ¥a4, and n equality restraints. 
The restraint vectors A“ corresponding to linearly independent (by assump- 
tion) equality restraints of the primal problem (A) are linearly independent. 
The variables y,, y,, ..., y, are not restrained by the requirement of non- 
negativity. The system A”), A”, ..., A™ therefore appears in the basis of 
any support program of problem {A), The basis of any support program 
Y=(Y.,-+-) Yas Years «+> ¥,) Of this problem comprises a vectors (as the number 
of restraints (8.4)) and, besides the vectors A”, it contains n—m unit vec- 
tors —e,. Let £, be the set of indices of the unit vectors appearing in the 
basis in question, and /, the set of indices of the remaining unit vectors. 
Thus, 


Ty =(Sy) Syy oe ey Sq) 
Ey = (Saas Sates sacs aly 
Ey+i/p=({1, 2, ..., n}. 
Consider the restraint vectors A, of problem (A) for /€/y. 
The vectors A, A, ..., A,, constitute a linearly independent system, 
Indeed, the determinant of the corresponding matrix A=(A,, ..., A, ) coin- 
cides, up to the sign, with the determinant of the matrix A=(A", ..., A™, 


—e, yy re) —e, ) Of the restraint vectors of the support program Y of prob- 
lem (A). This can easily be verified, if the determinant of the matrix A 
is expanded in the elements of the columns corresponding to the vectors 
—e,(J€ Ey). 

In restraints (1.2) of problem (A) we take x,=0for jE€F£, Then, in view 
of the linear independence of vectors A, (j€/,) restraints (1.2) define the set 
of numbers x,(/€/y). 


roe Ky = Xj, f= 1, 2, .-., m. 


We denote by x,, d=1, 2, ..., m J=1,2, .... 4, the corresponding coefficients 


in the expansion of the restraint vectors A, in terms of the vectors of the 
linearly independent system 4,, ..., A The following formulas can 
easily be verified (see, e.g., (1.15)): 


s ° 


m 
x.=> bey, (8.5) 

reas 
xy= > a, 7F ius (8.6) 

w=! 

i= 1, a. eaoy M, jJ=1, 2: , a 
Here 
lhe in = I eis, 7" (i, p=, 2, sees m),. 


Thus, to each basis of problem (A) there correspond m linearly independent 
vectors A,, i=1, 2, ..., m,and sets of numbers x,, and x,, satisfying rela- 
tionships (8.5), (8.6). 

We now compute the coefficients in the expansion of the restraint vec- 
tors and the constraint vector of problem (A)in terms of the basis vectors 
of support program Y. The restraint vectors of problem (A) not appearing 
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in the basis are the vectors e, with /€/,. The constraint vector of problem 
(A) is the vector C whose components are the linear-form coefficients of 
problem (A). 

Let Y,;=(y,, +--+) Yin) be the coefficients in the expansion of the vector —e,, 
(i= 1, ..., m)in terms of the basis vectors, and Y,=(y,,, ..., Y.,) the basis 
components of the support program Y of problem (A)— the coefficients in 
the expansion of vector C in terms of the basis vectors, 

By definition, 


—e,, =AY, 1=1, 2, 0... m, (8.7) 
C=AY,, (8.8) 
where ” 
A=(A™, ..., A™, —~es ay - 6, ). 


For the sake of convenience we interchange the rows of matrix A and 
write 


a5, Gn, 9 . +O 
185 MS, 0 - 0 
A=|| “ss Cus, 9 Ne 
m+ MSmer 0 
Qs, An, 0 —1 
From (8.7) and (8.8) we obtain 
Y,u— A" ¢,, ime l, 2, 20. m, (8.9) 
Y,=A7'C, 


It is only natural to assume that the components of the vectors e, i=1 
2, ..-, m, and of C in (8.9) and (8.10) are arrayed in the same way as the 
rows of the matrix 4. 

Direct matrix multiplication will readily show that 


e, en, O 0 
Cy Qo Dc 0 
A*=|1e oe 0... 0 (8.10) 
Cy may ee Ly ayy 0 
_ A $e. Sofas, 1a : . : ae = 
Here 
ea l= ais, Im", i=], 2, eee, mm; w= 1, 2; eeey ms 
m 
om = Qs Cy, $= 1,2,...,m,p=m+1,m+2, ... WM. (8.11) 


We rewrite relationships (8.9) and (8.10) using the explicit expression 
for the matrix A7! We have 


Jig lip i= 1,2, cosy M, J= 1,2, eee, A; (8.12) 
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PRIN for j=1, 2, ..., m; 

Ys=\ om (8.13) 
Poni ey OC Sanh RE ae 
=1 


Comparing (8.6) and (8.11) we obtain 
f=:1, 2, ..., mm p=m+1, m+2, ..., 4. 


Hence, applying (8.12) and (8.13), we obtain 


Y ip = — *isys i=1, 2, ooo, M, po=m+l, coe, A, (8,14) 


Yop = 3 ony Ms, — Caps po=m+l, m+2, oo, A, (8.15) 


We shall now find relations for the relative evaluations of the restraint 
vectors of problems (A) and (A), To distinguish between the evaluations of 
the restraint vectors of these problems, we shall modify A with super- 
scripts X and Y, respectively. 

The evaluations of restraint vectors are expressed in terms of the cor- 
responding linear-form coefficients and the coefficients in the expansion 
of these vectors in terms of the basis. According to the general rule of 
computation of the relative evaluations of restraint vectors, we have 


AY) =D Xu sls, — ps JEEY; (8.16) 
y=1 
AI) =D Vin b+) Vi, mrypoms sy — mes; i=1, 2, ..., m. 
w= pai 


By assumption, however, in problem (A), 6,=0 for j>m. Therefore, denot- 
ing Aw) by A”, we obtain 


m 
AM =P yb,  1=1, 2, 2.0) a 
w=) 


Further, applying relationships (8.12) and (8.5), we obtain 
AY) = — x; i= 1, 2, 02-5 mm. (8.17) 


From (8.15) and (8,16) we obtain 
AM =u, A=m+I,*..., M (8.18) 


Formulas (8.17), (8.18), and (8.14) enable us to establish the complete 
equivalence of all the stages in the solution of problem (A) by the simplex 
method and of problem (A) by the dual method. 

The parameters y,, in (8.18) are the basis components of a support 
program of problem (A) restrained by the requirement of nonnegativity. 
Therefore, according to (8.18), AM)>0 for /EF, or /€/,, This indicates that 
the restraint vectors A, j/€/, constitute the basis of a pseudoprogram of 
problem (A) with the basis components x,, i=1, 2, ..., m. 

Thus, to each basis of the support program Y of problem (A) there cor- 
responds a basis of the pseudoprogram of problem (A). The indices of the 
basis vectors and of the basis components of the pseudoprogram constitute 
the set /, defined by the basis of the support program Y of problem (A). 
The transition from one support program of problem (A) to another corres- 
ponds to a transition from one pseudoprogram of problem (A) to a succes- 
sive one, The introduction of the vector —e,(j€/,) into the vector of the 
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support program of problem (A) corresponds to elimination of the vector A, 
from the basis of pseudoprogram of problem (A), 

In problem (A) we minimize the linear form L(Y). Hence, the optimality 
test of the solution of problem (A) has the form 


AW) <0; JEly 
or, equivalently from (8.17), 
x, = 0; é=1, 2, ..., m. 


The last relationships constitute the optimality test of the pseudoprogram 
of problem (A) corresponding to program Y of problem (A). 

Case (a) of the simplex method, when applied to problem (A), is thus 
equivalent to case (a) of the dual method, when applied to problem (A). 

Now let the support program Y of problem (A) not satisfy the optimality 
test. In this case some of the parameters AY) are positive. Let A@>0, 
The condition for unsolvability of problem (A) is: for some A®>0 all 
Yy. <0, p=m+1, ..., 2. Formulas (8,17) and (8.14) show that with reference 
to problem (A) the requirement of unsolvability is stated as follows. To 
some negative basis component x,, of the pseudoprogram there correspond 
nonnegative coefficients x,, for all jE Ey. 

Hence, case (b) of the simplex method for problem (A) coincides with 
case (b) of the dual method for problem (A). 

Finally, let Af>0 and the unsolvability requirements not be satisfied 
(case (c)). Interms of problem (A) this indicates that the pseudoprogram of 
problem (a) defined by program Y of problem (A) has negative components 
and to each of these correspond negative x,,, In other words, we have 
case (c) of the dual method. 

Let us now analyze a single iteration in the solution of problem (A) by 
the simplex method. We introduce into the basis some vector —e,, for 
which AY)>0, The vector A, is then eliminated from the basis ot ‘pseudo- 
program of problem (A), Formula (8.17) shows that the vector A,, elimina- 
ted from the pseudoprogram basis corresponds to a negative basis com- 
ponent x,,. 

According to the simplex method, the vector —,,(s,€£y) on which 


6, = min Yor — Yot 
a > a ri Ort 


is obtained is eliminated from the basis of the support program of problem 
(A). 

According to (8.18) and (8.14) the preceding condition indicates that the 
vector A,=A,, on which 


=— 
= aa 


Xj <o 


§, = min ( “) NG 


x, J XrR 


is obtained is introduced into the pseudoprogram basis of problem (A), We 
have thus established complete equivalence between the application of the 
simplex method to the dual problem and of the dual method to the primal 
problem. 

8-2. When discussing the two algorithms of the dual method, we com- 
pared the bulkiness of computations in a single iteration of this method 
with the bulkiness of those in one iteration in the simplex method. We saw 
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that the computational procedure of the two methods are close in form, and 
the number of operations entailed in a single iteration is of the same order 
in the two methods. We remarked also that in general we have no informa- 
tion which would enable us to compare the number of iterations required 
for solving a problem by either of the two methods, 

Taking into account the specific features of the two methods discussed 
in Chapters 4 to 6 wecanchoose one of the two in each concrete case, de- 
pending on the problem (or the class of problems) in question, 

When choosing a method, considerable attention should be given to the 
determination of the initial support program. If, for example, the initial 
support program of the dual problem is obvious, whereas the initial support 
program of the primal problem is to be determined following the general 
rules, the problem should, obviously, be solved by the dual method. 

In the following we shall discuss another feature of the dual method 
suitable for solving a wide class of problems, 

The necessity often arises of solving several linear-programming prob- 
lems differing in one or more restraints. 

The exact statement of complicated applied problems studied by linear- 
programming methods generally leads to the solution of an entire series 
of problems differing in just a few restraints. The point is that when dis- 
cussing new problems without any previous experience as to how they should 
be attacked, we are often unable to establish from the start all the re- 
straints to be satisfied by the solution. An analysis of a tentative solution 
of the problem points to additional restraints which should be introduced. 
This process is often repeated more than once. 

Similar difficulties arise when solving classical linear-programming 
problems. For example, when tackling a transportation problem we often 
disregard, in the first stages, the limited carrying capacity of the commu- 
nication routes. Moreover, the carrying capacities of the communication 
routes often change intime. It is natural to require that program adjust- 
ment involve less operations than the initial determination of the transporta- 
tion network. 

The necessity in solving a series of linear-programming problems dif- 
fering in a single restraint arises when the integer programming method 
is applied. 

In all these cases it is, obviously, inadvisable to start solving the prob- 
lem anew when some new restraints have been introduced, We must learn 
to utilize the information stored in the optimal program of the initial prob- 
lem effectively. The dual method can be successively employed to this 
end, 

Let the solution of an initial problem (we shall again call it problem (A)) 
yield the optimal support program X*=(x', ..., x,) with the basis A,, ..., A 
We now have a new problem differing from problem (A) in the additional 
restraint 


Sm 


n 
D Fm41 7%7 ones (8.19) 
f=1 

or, equivalently, 


Rn 
p2 Qn+i, 7 X;+Xnay = Oasys 
21 


xno, = 0. 
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We shall refer to this problem as problem (C). We now give problem (C), 
dual with respect to (C). 
Minimize the linear form 


e m+ 
L(Y)= Dom 
é=1 
subject to the conditions 


m 
Dy isi tomes, jVnrri = Cy j= 1, 2, eaeg n; 
s=1 


0. 
If Yau 


n 


p> a et ae (8.20) 


the vector X* obviously solves problem (C). 
Now let condition (8.20) not hold, i.e., 


pa pu ees 
We shall solve problem (C) by the dual method. 


Suppose that problem (A) has been solved by the second simplex or second 
dual algorithm. In either case, the last principal tableau contains 


(a) the optimal program X*=(x;,...,x,)0f the primal problem, 

(b) the optimal program Y*=(y;,...,ym) of the dual problem, 

(c) the matrix |le;;i,=(As,, As,---,4s,)7', the inverse of the optimal-basis 
matrix, 


From the restraints of problem (C) we see that the vector 


Y* = (Yi, ++ Yims 0) 


is its support program with the basis A,,,...,Ase, 4,4, Here, by definition, 
ApH (Oyj eee rnp Smarr J=L Qe ret, 
Ans,=(0, 0,..., 0, 1). 


Hence, when solving the problem (C) by the dual method, the vector Y* 
can be taken as the initial support program, The corresponding principal 
tableau is easily constructed from the last principal tableau of problem (A). 
It is left to the reader (see Exercise 10) to verify the following relation- 
ships: = ; : 

RP Se (Xiph epihe Rnea)s (8.21) 


where 
Xn+1 = — py Omar, *l + On433 
=1 


Y*=(%, eee Von 0); (8.22) 

ej = ej, i, j= i, 2, 8 M; (8.23) 
€i.m+1 = 0, 1=1, 2, eee , mm, (8.24) 
envs=— & CM Gmar sy, J 2, wee sm (8.25) 
Oni, m+i=, (8.26) 


The bar indicates that the parameters refer to problem (C). 

The optimal program of the complex problem is generally obtained after 
few iterations of the dual method, This is due to the fact that program Y* 
is invariably close to the solution of problem (C), 
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8-3. When solving linear-programming problems, we may often essen- 
tially reduce the number of iterations if the process of solution can be ter- 
minated on a program for which the linear form differs from its optimal 
value by at most some predetermined value, Itis, therefore, necessary 
to be able to estimate the decrease in the value of the linear form arising 
when the optimal program is replaced by some intermediate program, This 
estimate can be obtained by solving the primal and the dual problems simul- 
taneously, or, equivalently, by solving the problem by simplex and dual 
methods simultaneously. 

Consider a pair of dual linear-programming problems, Let X=(x,,...,«,) 
and Y=(y,,...,¥,) be some support programs of these problems. Let V be 
the maximum value of the linear form of the primal problem, We know 
that 


Hence 7 
V—L(X)<L(Y)—L(X). (8.27) 


Thus, givensome program of the dual problem, we may estimate the 
departure of the linear form of the primal problem on an intermediate pro- 
gram from its optimal value. According to the first duality theorem, the 
optimal values of the linear forms of the dual and the primal problems 
coincide. The estimate (8.27) is, therefore, more exact the closer Y is 
to the solution of the dual problem, 

Suppose now that the primal problem is simultaneously solved by two 
methods, i.e., the simplex method starting with program X, and the dual 
method starting with program Y, Asa result, we obtain two sequences of 
programs 

Re XM eRe VMs Weck Ve: 


From (8.12) we have 
n k m k fn ~ 
6,= ee c jx <= m by; _— Pp» 6x = Oy: 


If 5, falls within preset limits, the process of solution is terminated and 
program X,is taken as the optimum, Otherwise, we should successively 
pass to new pairs of programs until 6, becomes less than the present value, 
This combined method is particularly effective when applied to 
linear-programming problems with numerous variables and restraints, 
There is still another advantage to the combined method, Assume that 
the optimal program computed by the simplex method is degenerate, In 
this case, to establish the optimality of the program we should perform, 
generally speaking, several iterations. If the problem is solved by the 
combined method, the optimality of programs of the dual problems is de- 
tected directly when the values of the corresponding linear forms coincide. 
We now give a geometrical illustration of the combined method. 


Example. Maximize the linear form 
L (X) = 10x, +8x, + 7x, + 16x, + 21x, 


subject to the conditions 


4x, + 2x, +5x,+ 10x, +5x,=6, 
9x, + 10x, + 12.5%, + 18x, + 16.5%, = 14, 
x= 0, PHA dese oe 
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Solution. In Chapter 4, 4-3, this problem was solved by the simplex method, In 2-8 of the present 
chapter the same problem was solved by the dual method, In both cases, the solution of the problem was 
illustrated by geometrical constructions in the three-dimensional space, 

Without repeating the considerations of the preceding articles, we shall apply the results in order to plot, 
on a plane, the successively contracting limits of the deviation of the current values of the linear form from 
the optimum. It must be kept in mind in the process that the value of the linear form of the dual problem 
on any program coincides with the value of the linear form of the primal] problem on the corresponding pseudo- 
program. 

We cut the polyhedral cone spanned by the augmented restraint vectors of the problem by plane H passing 
through line Q. All subsequent constructions are made in this plane (Figure 6. 9). 


FIGURE 6. 9 


The polyhedron A,A,A,A,A, is the section of the cone formed by plane H, The vertices A,of the 
polyhedron belong to the conical edges corresponding to the augmented restraint vectors Ay The planes 
If, II’, If” (images of the support programs X, X’, X" of the primal problem) meet plane H along lines 
A,Aw A,Ay,A,A,, Tespectively, The planes Il, If’ (images of the support programs Y, Y’ of the dual prob- 
lem) meet plane H along lines A,A,, A,A,, respectively. The ordinates of the intersection points L, L’, 
L’, L, L’ of these lines with line Q define the values of the linear forms of the primal and the dual prob- 
lems on the corresponding programs. 

We see that the values of the linear form on successive programs of the primal problem increase, 
whereas the linear form of the dual problem decreases on each successive program. At point L”asL’ cor- 
responding to the optimal programs of the primal and the dual problems, the linear forms of the two prob- 
lems coincide, 

Observe that for m==2 this m-dimensional variant of the second geometrical interpretation of a dual 
pair is simpler and more convenient than the (m-+1)-dimensional variant which is usually considered, 


EXERCISES TO CHAPTER 6 


1, Prove necesssity of the optimality test stated in 1-1 for the case when the support program of the dual 
problem is nondegenerate, 
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2, Prove that the distance of point X=(x,, 2X, ...X,), where 


Xe XX ht» j=Sp (=i, 2, Sysce fie 
a fa Xt, j= t 
0 in other cases, 
from the hyperplane 


L(X)=L (X,) 


(X,is a pseudoprogram of problem (1, 1)-(1,3) with the basis A, ooo, A.) is equal to 


| xt Ag] : 


V 34 


3, Prove that problems (3, 4)~(3,6) and (3,1)—(3.3) constitute a dual pair, 

4, Prove that the vector Y=(y,, Yg,--..Y,) is the kernel of a support program of problem (3. 4)~(3, 6) 
if and only if conditions (3.8) are satisfied and among restraint vectors Ay, j=1, 2,...,”, there are m linearly 
independent vectors satisfying relationships (3. 12). 

5. Prove that a support program (Y,2Z’, 2”) of problem (3. 4)—(3.6) is nondegenerate if and only if its 
kernel Y satisfies the condition 


A;=(Ay Y)—c, #0, i¢ly. 


where Jy is the set of indices of the basis vectors of the kernel Y of a support program of the dual problem 
(3. 4)-(3, 6). 

6. Pseudoprogram X of problem (3.1)~(3.3) and kernel Y of a support program of problem (3..4)-(3, 6) 
correspond to each other, if their components are related by (3,14), (3,15). Show that the vector Y corres- 
ponding, in this sense, to pseudoprogram X, which is a feasible program of problem (3, 1)—(3.3), is the kernel 
of the optimal program of the dual problem (3, 4)-(3, 6). 

7. Let X be a pseudoprogram of problem (3, 1)-—(3,3) and Y the corresponding kernel of a program of the 
dual problem (3, 4)-(3.6). Prove that 


where 


{ % for A,;=0, 
YB, for Ay<0. 


8. Let Y be a degenerate support program of the dual problem (1,4)—-(1.5) with m-+v parameters 4, 
equal to zero, Construct an example of a problem with Cr, +y bases of support program Y, 
9, Let A,, A,....,A,. be a system of linearly independent restraint vectors of problem (1,1)-(1.3). 
m 


We limit the variables x, whose parameters Ay= 2 C, Xiy—Cy are negative, by the number M>0 and 
=1 


solve the resulting bounded problem by the dual method. Let E be the set of indices of the extrabasis variables 
of the solution of the bounded problem, Prove the following two propositions; 
(a) there exists a number M,<oo such that for all M>M,, with pn y#0, the linear form of problem 
€ 


(1.1)-(1. 3) is unbounded in its domain of definition; 

(b) there exists a number M,<o such that for all M>M, unsolvability of the bounded problem implies 
unsolvability of the initial problem (1, 1)—(1. 3). 

10, Verify the validity of relationships (8,21)~(8,26) introduced in 8-2 in connection with the possibility 
of solving several linear- programming problems differing in a single restraint 

11, Applying the technique given in 7-6, 7-7, determine the initial support program of the problem 
which is dual with respect to the problem requiring maximization of the linear form 


L (X) = 4x, + 3x, + 10x,— 2x, 
subject to the conditions 
3x, + 2x,—x,+5x,— 2x,=8, 
Xg+3x,+6x,+3%,=—15, 
2x; —xX, +x, —2x,=0, 
xj>0, j=l, 2, 3, 4, 5 
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12, Solve the problem in Exercise 11 following the first and the second dual algorithms, 

13, Given the optimal program of the problem in Exercise 11, compute the components of the solution 
of the dual problem. 

14, Solve the problem in Exercise 11 subject to the additional restraints 


x2, f=, 2, 3, 4, 6 


15, Draw a block diagram of the gradient method for computing a support program of a linear-program- 
ming problem from any feasible program. 
16. A linear-programming problem calls for the maximization of the linear form 


L(X)=— (6 +1) 1 + (@—8) 4, +-(a@ +6—c—1) x, (6 +O—2) x + C5 + X_ toy + Xe 


subject to the conditions 


Xi+ X, + ax, + bx, +0cx%,+4%, = 10, 
—%, + 4x, + OX, +x, +6, +x, =5, 
AX, + bX, +C6X,+ XetXs +x,=20, 


x20. J=l, 2, 3, 4, 5. 


Determine the range of the parameters a, 6, and ¢ in which pseudoprogram X=(0, 0, 0, 0, 0, 10, 5,— 20) 
leads to cases (a), (b), and (c), respectively, 
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Chapter 7 
THE HUNGARIAN METHOD 


The simplex method is applied to the primal problem only and the dual 
simplex method deals only with the dual problem. We will now discuss a 
method which deals with both problems of the dual pair. In the method 
of successive residue contraction [known as the Hungarian 
method] the optimal program is found by moving over vectors with nonne- 
gative components. The rules of transition from one vector X to another 
are provided by contracting, from step to step, the differences between 
the right- andthe left-hand sides of the quality restraints of the primal prob- 
lem written inthe canonicalform, These differences arecalled residues, 
and the Hungarian method is thus referred to as the method of suc- 
cessive residues. It is proved that after a finite number of steps 
these difference will vanish or unsolvability of the problem will be estab- 
lished, 

The principal concepts of this method were first advanced in 1939 by 
L.V. Kantorovich, who used the method to solve particular linear-pro- 
gramming problems /61, 67/. L.V.Kantorovich does not consider the dual 
problem, but using decision multipliers (see Chapter 3, 5-1) defines its 
optimal program. 

An independent description of this method was given in 1956 by three 
American authors /52/. The history of this method in the American litera- 
ture is as follows. First, on the basis of some preliminary considerations 
contained in one of the older works of the Hungarian mathematician Eger- 
vary /123/, the dual simplex method was extended for the transportation 
problem /70, 111, 76/. In honor of Egervary the method was then called 
the Hungarian method. In /52/ the Hungarian method for the solution of 
the transportation problem was extended to the case of the general linear- 
programming problem. 

The material in this chapter is presented according to the following 
outline. 

In § 1 we describe the basic concepts of the method and give general con- 
siderations concerning the computational procedure, In § 2 the theoretical 
principles of the method are illustrated by two simple examples. In § 3 
we give a geometrical interpretation of the Hungarian method. In § 4 the 
method is extended to linear-programming problems with bilateral re- 
straints. The algorithm of the method is disccused in §5. Here the com- 
putational procedure of the Hungarian method is presented in application to 
problems in canonical form and problems with bilateral restraints and the 
discussion is elucidated by suitable examples. 

In the last section, §6, we discuss the revised Hungarian method— the 
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method of bilateral evaluations, This revised method often makes it pos- 
sible to obtain and evaluate an approximate solution of a linear-program- 
ming problem after appreciably fewer computations than in all other 
methods. 


§ 1. General outline 


1-1. We write the linear-programming problem in canonical form. 
Maximize the linear form 


L(X)= Sox, (1.1) 
f= 
subject to the conditions 
¥a,,x,=5, i=, 2, oo, M, (1.2) 
j=) 
x, = 0, jJ=1, 2. oo ey Mt, (1.3) 


The primal problem (1.1)—(1.3) will be denoted by (A). Problem (A) dual 
to problem (A) is to minimize the linear form 


(y= Boy, (1.4) 


subject to the conditions 


Te (1.5) 


We shall assume all the components of the constraint vector B=(b,,),,..., 
b,,)' to be nonnegative. Clearly, every linear-programming problem can be 
reduced to an equivalent problem satisfying this condition. 

.The discussion of the method is simplified if we introduce the so-called 
augmented andauxiliary problems. 

For brevity, we shall denote the augmented problem by (8), Problem (8) 
is to minimize the linear form 


ae (1.6) 
subject to the conditions 
Sait ter= by i=, 2, 005g M, (1.7) 
x, = 0, jJ= 1, 2, 00 0, ft, (1.8) 
e,>0, i=1, 2,...,m, (1.9) 


Let Y=()y,, Yay +--+ 3m) be a feasible program of the dual problem (A) (not 
necessarily a support program). The methods for finding an initial program 
of the dual problem are discussed in Chapter 6, 7-2. 

We select the restraint vectors A, j=1,2,...,2, such that 


Dy tui= or (1.10) 
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The set of indices of these vectors is denoted by Fy. To each program Y of 
the dual problem (A) we can associate an auxiliary problem (Cy) which is 
obtained from the augmented problem (8) when additional restraints are 
imposed: 

x,=0 for /¢€E£,. (1.11) 


The auxiliary problem (C,) is, thus, to minimize the linear form 


m 
p2 &; (1.12) 

=1 

subject to the conditions 

=b,, 1.13 
joe, ste i ( ) 
x,20 for JEé£,, (1,14) 
e, = 0, i=l, 2, noo, M, (1.15) 


When solving problem (A) by the Hungarian method we shall require the 
dual of the auxiliary problem (Cy) also. We shall denote the dual auxiliary 
problem by (C,). According to the general rules of construction of dual 
problems, problem (C,) is stated as follows. 

Maximize the linear form 


m 
>» Oy (1.16) 
subject to the conditions 
m 
tii SO for /€E,, (1.17) 
=1 
Ppl, i=1, 2,...,m. (1.18) 
The restraint vectors of the auxiliary problem (C,) are the vectors 
Aj, Ajay seer Af Cy) gy -+-1€, Where the indices j/,,...,/, constitute the set &,, 
and the vectors e,,e,,...,e, form the complete set of m-dimensional unit 
vectors. 
Let the components of a feasible program of the auxiliary problem (C,) 
be E/E Ly), e,(é=1, 2,...,m). To every program let there correspond some 
n-dimensional vector X=(x,,...,x,), where 


x,=6, for ae 


x,=0 for /GEy. (1.19) 


Generally speaking, vector X with nonnegative components defined by (1.19) 
is not a program of primal problem (A). From (1.13), (1.15), (1.19), 


by — Yayx= 020. (1.20) 
=1 


The components e, of a feasible program of the auxiliary problem thus de- 
fine the residues of the i-th restraint in (1.2) when the vector X=(x,,...,*,) 
is substituted in (1.20). The optimal support program of the auxiliary 
problem has at most m positive components (§,, «,. The number of posi- 
tive components of the corresponding vector X is a fortiori at most m., 

We shall call the vector X corresponding to the optimal support program 
of some auxiliary problem (C,)) aquasiprogram of primal problem (A): 
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The basis vectors of the solution of the auxiliary program form the quasi- 
program basis of problem (A), The basis By, of the quasiprogram is 
constructed of some restraint vectors of problem (A) and some unit vectors. 
If the quasiprogram basis contains no unit vectors, the quasiprogram is a 
feasible program of problem (A) and, as we shall see in what follows, also 
its solution. 

The vector E=(e,,...,e,), where @,,...,& are the last m components of 
the solution of the auxiliary problem, will be called the residue vector 


m 

corresponding to quaSiprogram X, and the sum e,= De of the components 
s=1 

of vector E will be called the residue of the quasiprogram, 


Optimality test. A quasiprogram X of problem (A) is its optimal pro- 
gram, tf all the components of the corresponding residue vector are zero, 
or, equivalently, if the quasiprogram with zero residue solves the problem. 


Proof. By construction, the components x, of the quasiprogram are non- 
negative. Moreover, by definition 


e,=0 for i=1, 2,...,m, 


Therefore, according to (1.20), the components of the quasiprogram satis- 
fy restraints (1.2) of problem (A). Hence, the quasiprogram is a feasible 
program of problem (A), According to the definition of program X=(x,,...,%,), 
for x, >0 we have 


m 
a.,V,;=C,}. 
>» pie 


We, therefore, have the following: 
n n m7 ™ " m 
PT = 2%) tii = 2 Psi) = PAT: 

By virtue of Chapter 3, Lemma 1.2, this result proves the optimality 
of program X of problem (A). Hence, program Y of dual problem (A) gen- 
erating the auxiliary problem (C,) is the solution of problem (A). This 
completes the proof, 

1-2, Let Y=(y,,...,y,) be a feasible program of the dual problem (A), 
Let auxiliary problem (C,) and dual auxiliary problem (C,) be associated 
with it. 

Let the optimal program of problem (C,) have the form 


(°, E*)=(5)., 5 elie | ae eg, ad oor 8) 
and the solution of problem (C,) be the vector 


M® = (pi, oes Pe) 


We introduce the parameters A, and bi, ys Pay ere 
Ay= Beaii-—ep J=1, 2,...,4, (1.21) 
6 = tite J=1, oa Saag Fhe (1.22) 


The solution of auxiliary problem (C,) determines a quasiprogram X of 
problem (A). The residue vector E and the parameter 6 are the defining 
characteristics of quasiprogram X, Depending on the residues e; and the 
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signs of the parameters 8), we have three cases: 
(a) All the components of the residue vector are zero: 


ep =0, f=1, 2,...,™ 


(b) There are positive residues and all the parameters 6 are nonposi- 
tive: 


m 
a= Be > 0 6° <0, j=l, 2,...,2. 
=1 


(c) The residue vectors have some positive components and at least one 
of the & is positive, 

In case (a), it follows from the optimality test that the quasiprogram is 
also the optimal program of problem (A). We shall show in the following 
that in case (b) problem (A) in unsolvable (problem restraints are incon- 
sistent), and in case (c) we may pass to a new quasiprogram with a smaller 
residue. 

1-3. To simplify the analysis of cases (b) and (c), we will note the 
variation of the sum of residues (the linear form of the auxiliary problem) 
when passing from program Y of the dual problem to the program 


y’=Y (6)=Y—6M’. 
The transition from program Y to program Y(#) is anelementary 
transformation of program Y, 


We compute the parameters A,(6) for program Y($). From (1.21) and 
(1.22) we have 


A, (9) = 24,9; (0) —¢,= A,—06), J=1, 2, .c.50 (1,23) 


The vector Y(0) is a feasible program of the dual problem (A) for all 6 such 


that 
A,(()>0 for j=1,2,...,2. 


We compute the value of the linear form of problem (A) on feasible 
program Y(6). We have 


L(Y ()) = Sow, (8) = 35,910 3 dp}. 


Applying the first duality theorem to the auxiliary problem (Cy) and to its 
dual problem (C,), we obtain 

m m 

Pa = dee 


Hence, ; 7 eS, 
Livy @)=Liy—8> ¢;. (1.24) 


We see that if §>0 and e; = Se; >0, the linear form of the dual problem 
=1 


decreases under elementary transformation of program Y into program Y (9). 
We shall now analyze case (b), where some of the residues e; are posi- 

tive and all 5) < 0, j=1,2,...,a. In this case we see from (1.23) that Y(6) 

is a feasible program of problem (A) for any 20. Equality (1.24) shows 

that problem (A) is not bounded below in the set of its feasible programs. 

Hence, according to Chapter 3, Lemma 1.3, the primal problem (A) is unsolvable 
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since its restraints are inconsistent. The conditions determining case (b) 
will be referred to in what follows as the unsolvability test of the 
problem, 

We now proceed to analyze case (c), where some of the residues e are 
positive and also some of the parameters 6 are positive, 

The greatest § for which Y(§) is stilla feasibie program of problem (A) 
is found from 


‘= mia (3): (1,25) 


6 ;>0 | 


Here the minimum is taken over the indices / for which 6, are positive. In 
case (c) there is at least one such index, Applying the definition of 5; and 
conditions (1.17) satisfied by p,, we obtain 


67<0, /€ Ey. 


Hence $6, is obtained for j¢£,. On the other hand, from the definition of 
the set £, the parameters A, are positive for /€ Ey. _Therefore, 6, > 0. 
We shall show that program Y’=Y(6,) of problem (A) leads to auxiliary 
problem (Cy-) with a smaller minimum sum of residues than for problem (C,). 
From the second duality theorem, for 


&, >0 
A 
we have 
e a e 
ZA = > ga ie 


From the preceding relationships we see that Ey. comprises, in particular, 
the subscripts j/ such that x,=§>0. Indeed, for these / 


A , (0) = A,—6,67 =0—8,-0 =0. 


The initial program of auxiliary problem (Cy-) corresponding to Y(6,) can, 

therefore, be constructed from the positive components of the optimal pro- 

gram of the previous auxiliary problem (C,). This essentially reduces the 

number of steps leading to the solution of the current auxiliary problem. 
We define j, from the relationship 


4;, e 
age 6), > 0. 
There may be several such indices, Clearly, vector A;, is one of the 
restraint vectors of the new auxiliary problem, since 


A, (8,) = A, —6,6', = 0. 


Let problem C, be nondegenerate (to this end it is sufficient to assume 
nondegeneracy of the augmented problem (8)), 

In § 5 we discuss when it is expedient to solve the auxiliary problem by 
the second simplex algorithm. According to the basic premises of this 
method, the condition 6, >0 for a nondegenerate problem indicates that the 
introduction of vector A, into the basis of the initial program of the auxi- 
liary program will decrease its linear form. The minimum of the new 
auxiliary problem is, therefore, strictly less than the minimum of the 


368 


previous auxiliary problem: 


p> er’ = 2 2° (6,) <2 eg. 


Thus, in case (c) we can find an elementary transformation of program 
Y of the dual problem into program Y’=Y(6,) which generates an auxiliary 
probiem (Cy-) with a lower minimum sum of residues. In other words, in 
case (c) we may pass from quasiprogram X to quasiprogram X’ witha 
lower residue, This sequence of operations constitutes one iteration of the 
Hungarian method, The procedure is repeated until we obtain either case 
(a) or case (b). 

1-4. We outline briefly the sequence of operations in one iteration of 
the Hungarian method. 

In every iteration we know the program Y of the dual problem (A) (this 
having been obtained at the endofthe previous iteration), The iteration 
starts with the construction of auxiliary problem (Cy). The solution of 
problem (Cy) gives a quasiprogram X of the primal problem (A). Quasipro- 
gram X is then investigated. From the magnitude of the quasiprogram 
residue and the signs of the parameters 6 we establish which of the three 
cases ({a), (b), or (c)) applies. 

If the conditions of the optimality test are satisfied, i.e., case(a), the 
quasiprogram obtained is the optimal program of problem (A) and the pro- 
cess of solution is terminated. If the conditions of the optimality test are 
not fulfilled, we must examine the signs of the parameters 6. If all 6) 
are nonpositive, the problem is unsolvable (case (b)). If some of the 6) are 
positive, case(c) applies anda new program Y’ of the dual problem, which 
is used in a successive iteration reducing the quasiprogram residue, is 
constructed, 

Each iteration of the Hungarian method is made up of several simplex 
iterations required to solve the corresponding auxiliary problem. We ob- 
serve that the feasible programs of any auxiliary problem are also programs 
of the augmented problem (8). The process of solution of the auxiliary 
problems thus involves successive transitions from one support program 
of the augmented problem to another, 

Transitions of support programs of problem (8) follow the simplex 
rules, This, however, does not mean that the solution of problem (A) by 
the Hungarian method involves determining the optimal program of problem 
(B) by the simplex method. Indeed, throughout every iteration of the Hun- 
garian method the vector to be introduced in the basis is chosen not from 
the entire set of restraint vectors of problem (8), but only from among the 


unit vectors 
e; VUj=1, 2, ..., m) 


and those A, for which 
m 


Here Y=(y,, ...,; ¥,) is a feasible program of problem (A) associated with the 
current iteration. Before starting with each iteration, the set of admis- 
sible vectors A, is restored using the new program of problem (A), 

The preceding considerations make it possible, in particular, to draw 
some conclusions concerning the finiteness of the Hungarian method, Indeed, 
for a nondegenerate augmented problem the transition from one support 
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program to another is accompanied by monotonic decrease of linear form 
(1.6). Therefore, in the process of solution we cannot return to a support 
program of problem (8) which has already been examined. The number of 
different support programs of the augmented problem is finite. Hence, 
the optimal program of a solvable problem (A) is obtained by the Hungarian 
method after a finite number of iterations. If the problem is unsolvable, 
its unsolvability is also established in a finite number of steps. 

If degenerate auxiliary problems are solved using the rules ensuring 
against cycling (see Chapter 5, 5-5), a degenerate augmented problem 
is also solved by the Hungarian method in a finite number of steps, 


§ 2, Examples 


2-1, We shall now illustrate the above by the example investigated in Chapter 4, § 3, by the simplex 
method and in Chapter 6, § 2, by the dual simplex method. 
Example 1, Maximize the linear form 


L (X) = 5x, —x,— 2x, + 5x, +5x,—x, 
subject to the conditions 


— 2x, +5x, +, = 10, 
4,— Xs +X, =i, 

x, +2x, + Xs =v, 
10x, — 3x, +%, = 15, 


x; 20, {/=1, 2, eee, 6. 


Solution, The above problem will be called problem (A). Its dual, problem (A), is stated as follows: 
Minimize the linear form 


L(Y) = 104, +a + 6yp + 154, 
subject to the condtions 
—2y, +Y¥st Yat 104, >5, 


5¥:,—4¥, +2y,— 34> 
“A 


=. 
=—2, 
Ys = «#5, 
Ya = 5, 
¥=—1 
Problem (A) corresponds to the following augmented problem (B): 


Minimize the linear form 


&, +e, +e, +& 
subject to the conditions 
—2x, +5x, +4, +e, = 10, 
X,— X, +X, +8, ==1, 
X,+2x, +x, +e, =6, 
10x, — 3x, +X, +e,=15, 


x, = 0, j=1, 2, eee, 6, 
e; >0, i=l, 2, 3, 4. 


It is easy to see that 
Y=(0, 5, 5, 0) 


is a feasible program of problem (A). We compute the parameters A;, i.e., the differences between the 
left- and the right-hand sides of restraints of problem (A). We have 


A=(A,, Ay, Aj, Ay, As, AJ =, 6, 2, 0, 0, 1). 


The set Ey in this case contains two indices: j=4 and j=5 (Ey={4, 5}). 
Auxiliary problem (Cy) generated by program Y is obtained from problem (B) when we set x,==0 for 
jeEy. In problem (Cy) we thus minimize ‘ 
pe," 
=1 
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subject to the conditions 


e, = 10, 
x, +e, =I, 
Xs +e, =6, 
e, =15, 


X,220, x50, 020, f=], 2, 3, 4. 


Without special computations, we see that the solution of problem (Cy) is given by the following set of 
numbers: 


X,=1, x,=6, 8,=—10, e,=0, e,=0, e,=15. 


Thus, 
(8&*, E*)=(1, 6, 10, 0, 0, 15). 


The initial quasiprogram is thus the vector X =(0,0,0,1,6,0). The residue of quasiprogram X does not 


4 
vanish (> e, =10+0 +0-+15= 25). Case (a) does not apply: quasiprogram X does not solve the problem, 
=1 
Problem (Cy), the dual of problem (Cy), is to maximize the linear form 


10p, +H, +6p, + 15p, 
subject to the conditions 


pea, p,0, prel, f=, 2, 3, 4. 


Since all the coefficients of the linear form of problem (Cy) are positive, problem (Cy) is solved by 


the vector 
M*=(1, 0, 0, 1). 


Following the recommended procedure, we find the parameters 


e 4 ‘ 
6.= 
j m, Gy hs 
for all f(/sel, 2, ..., 6. We have 
6* = (57, 35, 85, by, 8,, 5.) = (8, 2, 1, 0, 0, 1). 


Some of the components 85 of the vector’ 8* are positive, We thus have case (c). 
We compute 


§,= min es 
6;>0 7] 


We have 
5 5 
t=min (3 » 3, 2, =z. 


The vector A’=A(6,) is computed from 


A’ =A—6,6*. 
In our Case, 
a 19 121 3 
A =(0.2, a. 0, 0,5) 


We form a successive program Y’ of problem (A): 
Y’=Y (6) =Y—0,M*, 


ft ae: 5 
Y =(-F 5, 5,— ¥} e 


Program Y’ of problem (A) generates a new auxiliary problem (Cy’) which is to minimize the linear 


form ry 
z* 
=1 
subject to the conditions 
—2x, +e, = 10, 
*; + 2%, +8, = l, 
x; +X; +8, =6, 
10x, +e,=15, 


X, 20x00, x,>0; e;>0, i=1, 2, 3, 4. 
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We solve problem (Cy.) by the second simplex algorithm, It is natural to use the solution of problem 
(Cy) as the initial program of problem (Cy.). Already after the first iteration we find the optimal programs 
of problems (Cy-) and (Cy): 

(8°, E*’)=(1, 0, 5, 12, 0, 0, 5), 
M*=(!1, —8, 0, 1). 


The corresponding quasiprogram is 
X’=(I, 0, 0, 0, 5, 0). 


Its residue is ‘ 
a> &; =124+04045=17. 
=1 


To establish whether the problem can be considered unsolvable, we determine the signs of the components 
of the vector 6°’. In our case 
6°’ =(0, 10, 1, —8, 0, 1). 


We again have case (c), when an elementary transformation can be found to obtain a new program Y” 
of problem (A) and, consequently, a new quasiprogram of problem (A) with a lower residue, The transfor- 
mation parameter @, is 


19 1 ) 


0° = mi = = 
=m" 90° 5° S/T" 


We compute the vectors 
A’ =A’ (0), y’=y’ (9) 


and form the successive auxiliary problem (Cy). 

Subsequent computations follow the same procedure, The main results of each iteration are given in 
Table 7, 1. 

After the fifth iteration, we obtain the quasiprogram 


XY) = (48/23, 45/23, 101/23, 20/23, 0, 0) 


with a zero residue, According to the optimality test, X'Y) solves problem (A). The maximum value of 


linear form L (X) is 
L (X%) = 93/23. 


It is readily seen that Y) is the optimal program of problem (A). Indeed, 


LY) = 1 (XV) = 93/23. 


2-2, We now consider another example, which was described in Chapter 6, 2-6, to illustrate case (b) 
of an unsolvable linear-programming problem. 
Example 2, Maximize the linear form 


L (X) =5x, + 4%,+%; 
subject to the conditions 
X,+2x,—x, =x 2, 
4x, + 3x, +x, = 12, 
4x, +4x,+%,+%,+%,=10, 
4,0. j=1, 2, 3, 4, 5, 


Solution, The above problem will be called problem (A). The dual problem (A) is to minimize 
the linear form 
L (Y) = 2y, + 12y, + 10g, 
subject to the conditions 
9, +44, + 4y, = 5, 
24, +343 + 4y, 4, 
—Y + Y= 0, 
YWty,=0, 
¥,=>1. 


As an initial program of problem A we naturally take the vector Y=(0, 0, y,), where 


yy= max {5/4, 1, 0, 0, 1} 5/4. 
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TABLE 7. 1 


No, of 
itera - x) 
tion (v) i 
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TABLE 7. 1 (continued) 


A) 6* (¥) 


No. of 
jtera- 
tion(v)} f 
1 


2 


2154 
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For program Y we have 
A=(0, 1, 5/4, 5/4, 1/4). 
Hence, auxiliary problem (Cy) contains only one restraint vector of problem (A), namely vector A,. 
Problem (Cy) is to minimize 


3 
De 
f=1 


subject to the conditions 


x,+e,= 2, 
4x, -+e,=12, 
4x, +e,=—10, 


X,>0, e>0, i=1, 2, 3. 
It is easily seen, without special computations, that problem (Cy) is solved by the vector 
(2*, E*)=(2, 0, 4, 2). 
Hence the initial quasiprogram X is a vector with one nonzero component, 


X=(2, 0, 0, 0, 0). 


The residue of quasiprogram X is 6. 
Problem (Cy) dual to auxiliary problem (Cy) is solved by the vector 


M*=(—8, I, 1). 


We now compute the parameters of quasiprogram X required for passing to the next quasiprogram with 
a lower residue. We have 


6° =(0, —9, 9, 2, 1). 


mine! a2 
termine = 3° 


i 
A’ =A (8,)=(0, 9/4, 0, 35/36, 1/9), 
Y’ =(10/9, —5/36, 10/9). 


The program Y’ of problem (A) generates auxiliary problem (Cy.). The solution (&*, E*) of problem 
(Cy) can be taken as the initial program of problem (Cy.). After the first iteration we find 


(S*’, E*’)=(12/5, 2/5, 0, 12/5, 0), 
M* =(—4/5, 1, —4/5), 
X’ = (12/5, 0, 2/5, 0, 0). 


GB ~~ 
D> & =12/5, 
f=1 


6° =(0, —9/5, 0, 1/5, — 4/5). 


In our case 


Therefore we conclude that case (c) applies. We proceed with a successive elementary transformation of 
vector Y’ into Y"=Y' (6). We have 


+ 175 ” 
(= 3 A” =(0, 11, 0, 0, 4), Y”"=(5, —65, 5). 
The restraint vectors of the auxiliary problem (Cy) consist of three restraint vectors of problem (A), 
namely A,, A,, and A,, and the unit vectors ¢@,, é, @, We solve problem (Cye) starting with program 
(8*’, E*’). We obtain the quasiprogram 


X” =(2, 0, 0, 2, 0). 


> e, =2, 


6’*=(0, —1, —2, 0, —1). 


For quasiprogram X” 


The residue of program X” is positive, and all the components of vector 6°” are nonpositive, According 
to § 1 this indicates that the restraints of problem (A) are inconsistent, Indeed, subtracting the second 
restraint of problem (A) from the third, we find 
X,+%,+%,=—2. 
This equality cannot hold with nonnegative variables, Problem (A) is unsolvable. 
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§ 3. Geometrical interpretation 


3-1. The Hungarian method can be interpreted geometrically in the 
(m-+1)}-dimensional space of points U=(4,, u,, .... Ug, U_y,)- 

Problem (A) is solved by the Hungarian method by first choosing a feas- 
ible program Y of the dual problem (A). A geometrical analog of program 
Y is the hyperplane II containing the origin and passing above the cone K 
of problem (A). Hyperplane II contains the augmented vectors A; for JE E,. 
The intersection of hyperplane II and cone K is also a polyhedral cone. We 
denote this cone by Ky. Polyhedral cone ky is generated by augmented 
vectors A, for jE€E, For m=2, the cone Ky may either contain a single 
point (the origin), or else coincide with an edge of cone X or with a plane 
angle — a two-dimensional cone. Kn is precisely m-dimensional if Y is 
a support program of problem (A). 

To program Y of problem (A) corresponds auxiliary problem (Cy). The 
cone Ky of problem (C,) is defined by the restraint vectors A,(j€ Fy) and by 
the augmented unit vectors 

7= (6... 01,0, 6.450, 1), i=1, 2, ..., m. 


m 


Polyhedral cone K, is thus tangent to hyperplane u,,,=0 along the face con- 
taining the vectors A, (/€£,). Problem (C,) is thus solved by determining 
the lowest point of intersection of line Q and cone kK,. (In the auxiliary 
problem we seek the minimum of the linear form.) 

Clearly, problem (C,) is always solvable. Solving the auxiliary problem, 
we find the optimal programs of the dual pair of problems (C,) of (Cy). To 
the solution of problem (C,) corresponds hyperplane ay situated below the 
cone Ky, Let M*=(wi, ..., pi, —1) be the direction vector of hyperplane a, 
and Q, the lowest point of intersection of cone K, and line Q. The point Q, 
is the image of the solution of problem (C,). The vector Q, can be written 
in terms of basis vectors of the solution of problem (C,) in the form 


=F «Ast Seve, 


JéEy é=1 
Consider the point 
m = 
Py= > x A, (Py = Q—- > 8; é;) 
je Ey i=1 
on hyperplane u,,,=0. Through point P, let the line P parallel to the 
Ou,,,-axis pass. The highest point of intersection X of line P and cone K 
of problem (A) is the image of quasiprogram X. 
The distance between the points U’=(w',..., u 
is defined as 


) and U" = (u", ee 


m+ ea 


m+ 
lai. 
f=1 
Point X, the image of quasiprogram X, has the following extremum 
property: the point X is the nearest (in the sense of the preceding defini- 
tion of distance) to the highest point of intersection of the cone K and the 


line Q of all the points U=(u,, ..., ug, u,,,) Which 
(a) belong to cone K(x;>0); 
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(b) belong to hyperplane ITI: 


_ Cy J/=1, eee, A, 
a 2p) =C,, J€E,; 


(c) satisfy the restraints 


n 
a<=b, (3 Gj ;X) = ) e 
ai 


Also in the sense of the preceding definition of distance, the length of 
segment Q,8 (B is the point corresponding to the constraint vector) is given 
by the residue 


of quasiprogram 4X, 

Two cases are possible: point Q,, the image of the solution of problem 
(Cy), does or does not belong to hyperplane 4u,,,=0. In the former case 
quasiprogram X solves problem (A) (case (a)). Indeed, if point Q, coincides 
with point B, the constraint vectors belongs to the cone generated by the 
restraint vectors A, (/€ Fy). Hence, the vector B is a linear combination of 
vectors A, (/€£,) with nonnegative coefficients. A similar linear combina- 
tion of the augmented vectors A, (/E£,) gives the highest point of intersec- 
tion of Q and the cone K of problem (A). 

In this case program Y solves problem (A), and X is the optimal pro- 
gram of the primal problem. 

To analyze the case when point Q, does not belong to hyperplane u,,,=0, 
we consider the geometrical interpretation of the elementary transforma- 
tion of program Y of problem (A) into program Y’. 


Let Y*=(y,, Yur «++: Ye. —1) be the direction vector of hyperplane I]. The 
direction vector of hyperplane a, corresponding to the solution of problem 
(Cy), was denoted by M*=(pi, ..., wy, —1). 


Consider hyperplane s, which is the intersection of hyperplanes x, and 
u,,,=0 parallel to the Ou,,,-axis. The first m components of the direction 
vectors M* and M* of hyperplanes az, and a, are the same, The last com- 
ponent of vector M* is zero. 


The vector _ 
Y* (6) = yY*—6M* 
is the direction vector of hyperplane II(#). For 6=0 hyperplane II (6) coin- 
cides with Il. As 6 increases, hyperplane II(6) rotates about the intersec- 
tion S of hyperplanes II and ny. 

The hyperplane rotates (6 increases) as long as II(6) does not capture 
one of the augmented vectors A, (é£,). Let this occur for §=6,. Hyper- 
plane II’=II(6,) corresponds to a new program Y’ of the dual problem (A). 
This follows from the fact that none of the vectors A, (jE £,) can be above 
hyperplane II’, since 

(Y’ (6), A,) = 0— 68; > 0. 


Observe that the intersection of hyperplane x, with the coordinate hyper- 
plane u,,,=0 contains all the basis vectors A,(/€£,) of the solution of prob- 
lem (Cy). The corresponding augmented vectors A, belong to hyperplane S, 
the intersection of II and a,, and, consequently, to hyperplane II’ =I (6,). 
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Hyperplane x, passes below the cone K,. Hence, all the vectors A; Vé £,) 
lie on one side of hyperplane zy. The other restraint vectors A, (/¢£,) need 
not meet this condition, They fall on either side of x,. 

We shall consider two cases. 

First let all A, (/€E,) lie in the same halfspace relative to ay as the vec- 
tors A,V€ Ey). In this case hyperplane My separates the cone K and the 
line Q. Indeed, under our assumptions, 


(A, M*)=6) <0, j=l, 2, ..., 2. (3.1) 
On the other hand 
(B+heny, *)=(B, M) => dps =e* >0. (3.2) 
isi 


Vector B+dAe,,, defines the line Q. The first and the second equalities 
in (3.2) are obvious. The third equality follows from the duality theorem 
for the auxiliary problem, 

Inequalities (3.1) and (3.2) indicate that line Q and cone K do not inter- 
sect, In this case, §,=oo, i.e., none of the vectors A, (/¢£y) is captured 
by hyperplane II(0) for any finite value 6>0. 

Thus, if all the vectors 4; (j¢£,) fall on the same side of hyperplane xy 
as the vectors A; (VeéE,), the restraints of problem (A) are inconsistent 
(case (b)). 

If at least one of the vectors A,(/¢£y) and the vectors A, (/€ Ey) lie on dif- 
ferent sides of ny, we have case (c). Hyperplane II is rotated about S (the 
intersection of Il and x,) until some 4A, (¢£,) is captured. The correspond- 
ing hyperplane II’ gives a new program Y’ of the dual problem, which 
generates an auxiliary problem and a quasiprogram X’ with a lower residue, 
After a finite number of steps we obtain case (a) or case (b). 


32, We shall give the geometrical constructions illustrating the application of the Hungarian method 
to the determination of the optimal program of the problem solved in Chapter 4, 4-3 by the simplex method 
and in Chapter 6, 2-8 by the dual simplex method, 

In this problem we are required to minimize the linear form 


L (X) = 10x, + 8x, + 7x, + 16x, + 21x, 
subject to the following conditions 
4x, + 2x, + 5x, + 10x, +5x; =6, 
9x, + 10x, + 12.5x, + 18x, + 16.5x, = 14, 
x,20, J=1, 2,..., 5. 


Figure 7-1 shows the restraint vectors A, and the constraint vector B, the corresponding augmented vec- 
tors of the problem and the line Q passing through the point B parallel to the Ou,-axis, The figure, more- 
over, shows the polyhedral cone K generated by the vectors A, and the augmented unit vectors @ essential 
for the geometrical interpretation of the auxiliary problem, 

It is easily seen that the vector Y=(6,—2/5) is a feasible program of the auxiliary problem in which 
we try to minimize the linear form 


L(Y) =6y,+ 14y, 
subject to the conditions 


4y, + 9y, = 10, 
2y,+10y, > 8, 
Sy, + 12.5y, >7, 
10y,+18y, = 16, 
Sy, + 16.5y, => 21. 
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FIGURE 7,1 


v3 


FIGURE 7,2 
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Program Y reduces the second restraint to an equality. The set Ey comprises in our case only one index, 
j=2. Hence, the plane Mf), the geometrical image of program Y, contains the vector A,. The equation 
of plane Illy has the form 

6u,— 2 Us = Uy. 

The plane My extends above the cone K. Program Y generates auxiliary problem (Cy). The restraint 
vectors of problem (Cy) are Ay, é,, @, The polyhedral cone Ky corresponding to the auxiliary problem is 
shown in Figure 7.2, This figure also shows the cone K of the primal problem and the plane Fy — the 
image of program Y of the auxiliary problem, The line Q intersects the cone Ky at two points, The lowest 
point of intersection, Qy, of line Q and cone Ky lie in the plane My spanned by the vectors e, and Ay. 

The plane My is the geometrical image of the optimal program of problem (Cy), the dual of problem (Cy). 
Point Qy corresponds to the solution of problem (Cy). We resolve the vector Qy along the edges @,, €, Ay 
of cone Ky. Point Py is the projection of point Qy on vector Ay. The line drawn from point Py parallel 
to Ou, intersects the cone K of the primal problem at two points, The highest of these points, point X, is 
the geometrical image of quasiprogram X specified by the solution of problem (Cy). 


FIGURE 7.3 


The length of segment Qy,B (in the sense of the previous definition of distance), i.e,, the minimum 
value of the linear form of the auxiliary problem (Cy), is equal to e*— the residue of quasiprogram X. Since 
e* == 16/50, quasiprogram X is not the solution of the problem, The length of segment XPy gives the 
value of the linear form of the primal problem on quasiprogram X, L (X)=56/5, The plane xy intersects 
the coordinate plane u,=0 in the direction of vector A,. The plane Ty is, thus, defined by the vector A, 
and the Ou,-axis or, equivalently, by vectors A, and Ay. 

The planes My and xy intersect along the line Sy parallel to vector A,. We rotate the plane IIy about 
the line Sy in the sense of the arrow untilone of the augmented vectors A, is captured, The first vector 
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to be captured in the process is Ay. The corresponding position of the plane will be denoted by Ny. The 
plane My. spanned by vectors A, and A, corresponds to a new program Y’ of the dual problem (A), The 
vector Y’ is a support program of problem (A). Program Y’ defines a new auxiliary program, Geometrically 
this means that the plane Hy. defines the cone Ky, of problem (Cy,). The tetrahedral cone Ky, is generated 
by the augmented unit vectors e, € and the restraint vectors A, and Ay. 

The constructions of the second iteration are shown in Figure 7,3, As in the first iteration, we find the 
point Q)- corresponding to the solution of the auxiliary problem (Cy). The point Q,. lies above the plane 
u,=0. Hence, case (a) does not apply, The line of intersection of plane ay, with plane uy=0 extends 
along the vector A,. The direction of the vector A, and the Ou,-axis define the plane ex The restraint 


vectors A, (GE y- == {5}) lie on either side of plane ‘Thy? Hence, case (b) does not apply either, 


FIGURE 17.4 


Following the same rules as in the first iteration, we find the point X’, the image of quasiprogram X’ 
determined by program Y’ of the dual problem, In our case Sy., the line of intersection of planes Ty. and 
Ry, extends along the vector Ay. _ 

We rotate the plane II, about the line Sy,, the line of intersection of planes II), and xy,, until 
another vector Ay (igFy,) is captured. We thus obtain a plane Hy. corresponding to program Y” of the 
dual problem (A). The plane Il,» is spanned by vectors A, and A,, The vector Y” is a support program of 
problem (A). The cone Ky» of the auxiliary problem (Cy) is generated by vectors €,, eg, Ay, and Ay. 

Point Qy», the image of the solution of auxiliary problem (Cys), coincides with point B. The residue 
of quasiprogram X” is zero and quasiprogram X” is thus the solution of the primal problem. The construc- 
tions connected with the last iteration are shown in Figure 7,4, 
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§ 4, The case of bilateral restraints 


4-1, The Hungarian method, like other linear-programming methods 
discussed in previous chapters, can be extended to problems with bilateral 
restraints. How this is done will be shown below. 

Consider a general problem with bilateral restraints: maximize the 
linear form 


A 
L(X)= ey, (4.1) 
subject to the conditions 
a 
a,<x,<8,, j=, 2,..., 9%. (4.3) 
Here 
Aj=(Bip Bap eer Og)", B=(d,, by, ..+) by)% 


The rank of matrix (A,, A,,..., A,) is assumed to be m. 

Each variable x, of problem (4.1)—(4.3) can be restrained on two sides 
or on one side only. In the latter case, the second bound is assumed to be 
at infinity (a,=—oo or B,=co), Primal problem (4.1)— (4.3) will be de- 
noted as problem (A). In the following we shall deal with kernels of pro- 
grams of problem (A), the dual of problem (A). Without repeating the state- 
ment of problem (A), whichis given in Chapter 6, 3-2, we write only the condi- 
tions satisfied by a program kernel of this problem. 

The vector Y=(y,, y,, ---» Ys) iS the kernel of a feasible program of prob- 
lem (A) if and only if 


m 
24;,¥;—¢;20, if B,=00, 
A,= =1 

~) (4,4) 
Ait SO, if a,=—oo, 


It follows from (4,4) that when all the problem variables are restrained on 
two sides (a,#— oo, B,#0oo), any vector Y is the kernel of some feasible pro- 
gram of problem (A). To each kernel Y there correspond, generally speak- 
ing, several feasible programs of problem (A). If we choose from these 
programs that which minimizes the linear form of problem (A), we establish 
a one-to-one correspondence between the kernels and the programs of 
problem (A), 

Let L(Y) be the value of the linear form of problem (A) on the program 
corresponding to kernel Y. We have 


™ n 
L(Y) = Yoy,— DAyp (4.5) 
where a fet 
a, if A,>0, 
w= B, if A,<0. (4.6) 


Here, as usual, 
A,= p? Fiji — Cp. (4.7) 


Conditions (4.4) and equality (4.5) are derived in Chapter 6, 3-2, 
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4-2. We now extend the concepts of auxiliary and augmented problems, 
introduced in §1, to problems with bilateral restraints. 
The augmented problem is to minimize the linear form 


ze (4.8) 
subject to the conditions 
2 aie +8i= bp f=T, 2, eee, m, (4.9) 
c= | 
a,=x,<SB, &, = 0; (4.10) 


Ja, 2 én He PH 2 ck mM: 


This problem will be denoted as problem (8). LetY=(y,, y,,..., y,) be the 
kernel of a feasible program of problem (A). Let Ey be the set of indices / 
such that 


m 


Each kernel Y is associated with an auxiliary problem (problem (C,)), 
which is formed from problem (8) by adding restraints: x,=y, /€£,, where 
vy, are defined in (4.6). 

Problem (C,) thus involves minimizing the linear form 


m 
Ze (4.8) 
subject to the conditions 
2, a, x,+e,= 0)"; (4.11) 
a,Sxj=pi 0,20; (4,12) 
J€E,, i= 1, 2, oun) Mm. 
Here 
bY) = b, — > ViAiz- (4.13) 


J€Ey 


The restraint vectors of the auxiliary problem (C,) are some of the re- 
straint vectors A, of problem (A) and the unit vectors e,, ¢,,..., e,. The con- 


straint vector 
Y Y "4 
BY = (BM, BP, 2, OF) 


of problem (C,) is formed from the constraint vector B of problem (A) ac- 
cording to (4.13). ; 
Let problem (Cy) be the dual problem of (C,). Every program (&, &), 


E=(E), JELy B=(e,, yy e+) Em) 
of problem (C,) can be made to correspond to vector X=(x,, x,,..., x,) with 
the coordinates 
. =| Ep JEE,, 
Typ J€Ey (4.14) 


where y, are defined in (4.6). 
Clearly, the vector X satisfies restraints (4.3), and 


6, — 24; x, =e, >0. (4,15) 
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We now introduce some definitions. 

The vector X=(x,, x,,..., ¥,) Corresponding to the support program (&%*, &*) 
of any auxiliary problem (C,) will be called a quasiprogram of the 
primal problem (A). Any quasiprogram satisfies restraints (4.3) of prob- 
lem (A). According to (4.15), the parameter e; >0 specifies the residue of 
the i-th restraint of system (4.2) when quasiprogram X is substituted in 


this restraint, The sum e,;= Se; will be called the residue of quasi- 
f=1 


program X, 

Since e; 20, the condition e, =0 implies that all e, are zero. 

A quasiprogram with zero residue thus satisfies equalities (4.2) and is, 
consequently, a feasible program of problem (A). 

4-3. The process of solution of problem (A) by the Hungarian method in- 
volves transferring quasiprograms of the problem. We shall now give a 
condition sufficient for the process of solution to yield the optimal program 
of the problem: 

Optimality test. If the residue e, of quasiprogram X vanishes, X is the 
optimal program of the primal problem. 


Proof. As we have already seen any quasiprogram of problem (A) with 
a zero residue is a feasible program of the problem, The vector X is, 
thus, a feasible program of problem (A). Let program X correspond to the 
solution of problem (C,) where Y=(y,, y,, ..., Yxg) is the kernel of a feasible 
program of problem (A). Since the program X is formed from the solution 
of problem (C,) according to (4.14), we have 

A= 214,9,—¢,=0, if aj<x<B, 
A, 20, if x,=a,, 
A,< 0, if x,=6, 

According to the second form of the optimality test established in Chap- 
ter 4, §5, these conditions show optimality of program X. This completes 
the proof, 

Let Y=(y,, ¥y, +++) Yq) be the kernel of a feasible program of problem (A). 
Consider auxiliary problem (C,) generated by this kernel. Problem (C,) can 
be solved by the simplex method described for problems with bilateral 
restraints in Chapter 4, §5, Let the set of numbers §&) (/€&,), e, i=1, 

2, ..., mdefine a support solution of the auxiliary problem obtained by the 
simplex method. Then, according to the second form of the optimality 
test (Chapter 4, 5-3), there exists a vector M* = (1, Us, oe Um) such that 


m = Vv; if Wj <EI< Pp, 
D4, <= 0, if E, =p, (4.16) 
ici > 0, if E,=B, 


.( =i, if e>0, 
<1, if e=0. 

If we use the second simplex algorithm, the vector M’ is determined at 
the same time as the solution of problem (C,). We leave it to the reader 
(see Exercise 6) to show that M* is the kernel of the solution of problem (Cy). 

Let X=(x,, x,,..., %,) be a quasiprogram of problem (A) corresponding to 
the given solution of problem (C,), 
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From (4.14) we have 
a b), JE Ey, 4 17 
“4 Vp IGE, ( " ) 


where y,, equal to d, or B,, are defined in (4.6). Let 


8) = Dati, J=1, 2 Saas fn. (4.18) 


Three cases arise in the analysis of quasiprogram X depending on the 


value of its residue e = de and the relative signs of A, and 5; (see (4.7) 
and (4.18)). 


(a) The residue e&, of quasiprogram X is zero. 
(b) The residue e>0, and 


i{ =0, if A,<0, (4,19) 


<0, if A,>0. 


(c) The residue e>0, but conditions (4.19) do not hold. 
We shall consider each of these cases separately. 


If e, =0 (case(a)), then, according to the optimality test, quasiprogram X 
is the optimal program of problem (A). The other two cases are con- 
veniently investigated using the elementary transformation introduced in § 1. 

4-4. An elementary transformation of kernel Y is defined as 

Y (6) = Y—0M?, (4.20) 


or, equivalently, 


y,(0) =y,—Op;, d=, 2,..., m. (4.20) 
Let 


m 
A= Lay B—ep f=, 2.0, 0. 
Applying equalities (4.20') and the definition of A, and 8), we have 
m m 
jJ=1, 2,..., a. (4,21) 
According to (4.16) and (4.18), we obtain for /€E,: 


=Q for Ox <8, 


=0 for +#,=6, 
Hence, for any 620 and /E€L,, we have 


; of =O = if ace <Bp 
A, (6) = A,—66, = —68, = 0, if a,=X, (4.22) 
<0, if Xx,=B,. 


From the set of all elementary transformations (4.20) we choose those 
whose parameter §>0 satisfies 


AA, (0) 0, /tEy. (4.23) 


In other words, we take a set of nonnegative § on which A, and A,(6) have 
the same sign for all /€Z,. If A/>0, the j-th restraint (4.23) may be violated 
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only if §;>0, which occurs if 


p= L0. (4, 24) 
6; 

Analogously, if A,<0 the corresponding restraint in (4.23) is not satis- 
fied only for 6;<0. The value of § on which the j-th restraint is not satis- 
fied in this case is also given by (4.24). Hence, all the inequalities in sys- 
tem (4.24) are satisfied if and only if 


A 
6<6,= min > , (4,25) 
z>e / 
° 


where the minimum is taken over the indices / such that 
A ° 
—!>0, 8) #0. 
6, 
A : ae : 
* <0 or 8;=0, then, by definition, 6,=00, i.e,, the 
parameter 6 is not bounded above. 

In the following we shall confine the discussion to elementary transfor- 
mations with nonnegative values of the parameter 6 satisfying condition 
(4.25). 

Note that according to (4.22) and (4.23), with a,=—oo (B,=0o) 


A, (0) <0 (A, (6) =). 


If for any /€E,, 


Therefore, with the specified values of 6, the vector Y(6) is the kernel of a 
feasible program of problem (A) (conditions (4.4) are satisfied). 

Now let us note the change in the linear form of problem (A) under ele- 
mentary transformation (4.20) with a parameter (§>0) satisfying inequality 
(4.25). According to (4.5), the value of the linear form of problem (A) on 
the program corresponding to kernel Y(6) is given by 


L(Y) = Boy (8)— v, (6) A, (6), (4.26) 


where 


_) & if A, (6)>0, 
10={ By, if A, (6)<0. 


We now transform equality (4.26). If /E2, and A,(6)#0, then, from (4,22), 
vy, (0) =x, 


If now /¢E, then, by assumption, A,(6)A,>0. Hence, from the definition 
(4.17) of quasiprogram X, we again have y,(6)=x,. 
We may thus take 
y, (0) =x, for j=1, 2,..., a, (4,27) 


Applying (4.27), and also (4.20') and (4.21), we obtain 


m Rn m nr 
Ly (6))= 2 by, — 2 xA,—6 [2 bu; —2 xi}. (4,28) 
Since Y= Y(0), we have 
Brin BxAy=L (Y), (4,29) 
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The value L, of the linear form of problem (C,) on the program defined by 
the set of numbers § =x, /E£y, e, i=1, 2,..., mis given by 


Lo=e = Dei. (4.30) 


The value Z, of the linear form of problem (C)) on the program correspond- 
ing to kernel M* is given by 


2 124, *AI: (4.31) 


Here we used the statement of the auxiliary problem (4.8), (4.11), (4.12), 
equality (4.5), and also (4.22), 

In deriving (4.5), maximization of the linear form of the primal problem 
was required, It is easily seen (see Exercise 7) that (4.5) also applies 
when minimization of the linear form of the primal problem is required, In 
this case, however, y, are defined as 


Relationship (4.22) shows that in this case y,=x, We transform (4,31) 
applying (4.13) for dS": 


Dy x, Vaigi— DY xp. 


m m m 
L = b. —_ a. % ‘— x,6;= b.u7— 
. p2 Be p Ce, if pe 2, f / p> iP i¢ y i=i jeEy 


Further applying (4.18), we obtain 
ig= Yopt— Dd x,d5. (4.32) 


Since L, and L, are the optimal values of the linear forms of problems (C,) 
and (C,), respectively, we have from the first duality theorem (Chapter 3, 


Theorem 4,1), . 
Lo=Le. 


Hence, applying (4.30) and (4.32), we obtain 


e= Lel= 2 bpi— 2 x 84. (4,33) 
Using (4.29) and (4.33), we rewrite (4.28) as 
L(Y (8)) =L (Y)—bes, (4.34) 


where ze is the residue of quasiprogram X corresponding to the solution of 
the auxiliary problem (Cy). 

We emphasize that equality (4.34) was derived under the assumption that 
0<§<6,. 

4-5, We are now in a position to give a comprehensive analysis of 
cases (b) and (c) stated in 4-3. 

Consider case (b). According to (4.19), 6;#0 and A, have different signs 
for all /€Ey. Hence, in this case §,=00o. Therefore, the vector Y(#) is the 
kernel of a program of problem (A) for any §>0. Further, since the residue 
e, of quasiprogram X is positive and equality (4.34) applies for all 6>0, we 
conclude that the linear form of problem (A) is unbounded in the set of its 
feasible programs. Hence, also, the primal problem (A) is unsolvable 
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because its restraints are inconsistent. Case (b) thus indicates unsolva- 
bility of primal problem (A). 

Case (b) is sometimes called unsolvability test of the primal 
problem. 

In case (c) the residue e of quasiprogram X is positive and for some 
GE x 

/ ° 
—>0, 8; # 0, 
6; 

From (4.25) we have 


Aj 
0<6, rare <0, (4.35) 
Io 
where j, is one of the indices / on which 6, is obtained. Let Y’=Y(6,). We 
proved in the previous article that the vector Y’ is the kernel of a program 
of problem (A), and according to (4.33) 


L(y’) =L(Y)—6,e8. 
Hence, since 6, and e, are both positive, we obtain 
L(Y’)<L(Y). 


The transition to kernel Y’ thus decreases the linear form of problem (A). 

The kernel Y’ generates a new auxiliary problem (Cy-) which is solved to 
obtain a new quasiprogram X’ with a residue e?. 

We shall show that the set of numbers Xn JE Ey, &, i= 1,2, ..., m constitutes 
a support program of problem (Cy). Let /€&,, i.e., A, =A ,(6,) #0. From 
(4.22) (for /E£,) and (4.23) (for /¢£,), we find 


B, if A;<0 
Hence 
= bi — V2; ,=5;— p AP iy 
y’ 
The set of numbers x, /€ By, ef, i=1, 2, ..., mtherefore satisfies not only 


conditions (4.12), but also conditions (4.11) and is, thus, a feasible program 
of problem (Cy) (in restraints (4.11) and (4.12) the subscript Y is replaced 
by Y’). 
m If the restraint vector A, of the problem (A) is one of the basis vectors 
of the optimal support program x,, /E Ey, ef, i=1, 2, ...,m of problem (C,), 
then 8;=0. Therefore, 
A;=A,—8,6; =0, 


i,e., JEBy. Hence, the program x,, /€ Ey’, ef, i=1,... m of problem (Cy) is 
a support program. Its basis coincides with the basis of the support solu- 
tion of problem (C,). This support program is generally fairly close to the 
optimal program of auxiliary problem (Cy). Taking it as the first program, 
we substantially reduce the number of iterations required to solve prob- 
lem (Cy:). 

When solving a nondegenerate problem (Cy), the transition to quasipro- 
gram X’ lowers the residue: 


é 
25 <ed. 
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Indeed, from (4.35), 
A;,=A,, —0,87, =0. 


Therefore j,€ Fy, i.e., A, is a restraint vector of problem (Cy). Again, 


from (4.35) A ,83,>0. (4.36) 


From the definition of quasiprogram X it follows that 
“=| a = A> 

By» if A,,<9. 
Applying (4.36), we can rewrite the last equality as 
-{ a, if 69,>0, 

By if 6%,<0. 
Relationship (4.37) shows that the support program «x,, /€ Fy, ef, i=1, 2, ..., m 
of problem (Cy-) fulfills the requirements of case (c) of the simplex method 
(see Chapter 4, 5-5), and the vector A,, can be introduced into the basis. 
To verify this proposition, it suffices to see that problem (Cy-) involves 


minimization of the linear form and case (b) cannot arise in the process of 
solution: 


% fq 


(4.37) 


"i 
De, 20. 


é=1 
Since the vector A, is introduced into the basis, we obtain a new support 


m 
program of problem (Cy.), on which the linear form > e, assumes a lower 
§=1 


value than 


(by assumption, problem (Cy) is nondegenerate). Hence, the minimum 
value ef of the linear form of problem (Cy) is a fortiori less than the residue 
es of program X. 

In case (c) we can, thus, pass to a new quasiprogram X’ with a lower 
residue (this applies to a nondegenerate problem (Cy’)). 

4-6. The solution of a problem with bilateral restraints by the Hungarian 
method involves transferring its quasiprograms. 

The process of solution is made up of identical iterations. of which we 
shall describe one of these, say, the first. 

Let Y be the kernel of a feasible program of problem (A). As we have 
already observed, if all the variables of problem (A) are bounded on both 
sides, any m-dimensional vector can be taken as Y. 

The kernel Y generates auxiliary problem (C,). Without loss of 
generality, we may take all the components 4{") of the constraint vector B™” 
of problem (C,) to be nonnegative. The vector with the components §,=0, 
JE Ey e,—6", i=1, 2, ..., mis, therefore, a support program of problem (Cy) 
and its basis consists of unit vectors e,, e,, ..., ¢,. Taking this program as 
the first, we solve the auxiliary problem (Cy) by the simplex method (the 
second algorithm) for problems with bilateral restraints (see Chapter 4, 
§5, and Chapter 5, 7-3). 

After a finite number of steps, we obtain the support solution 


5%, JEE£,, ef, i=1, 2, ove, M, 
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of problem (C,) and at the kernel M*=(p{, ..., um) of the optimal program of 
problem (¢,). Applying (4.17), we form quasiprogram X and proceed 
to analyze it. 

If the residue e? of quasiprogram X is zero (case (a)), the quasiprogram 
X solves problem (A), Otherwise, the parameters 8j are fomed (A, have 
been previously computed while forming the set £,) and conditions (4.19) 
are checked. If these conditions are not satisfied(case(b)), problem (A) is 
proved unsolvable. In either of the two above cases the process of solu- 
tion naturally terminates. 

Now let e3>0 and let A, and 8) have the same signs for some /€By, i.e., 
A 8; >0 (case (c)). In this case a new kernel of a program of problem (A) is 
formed and we proceed with the next iteration. As the new kernel we take 
the vector 


Y’=Y—6,M*, 


where 6, is obtained from (4.25). 

The kernel Y’ defines a new auxiliary problem (Cy), The next iteration 
starts with the solution of this problem. As the basis of the first program 
of problem (Cy’), we take the basis of the support solution of problem (Cy). 

The next iteration is the same as the above with one of the following 
three outcomes: the solution of problem (A) is obtained (case (a)), unsolva- 
bility of problem (A) is established, or auxiliary problem (Cy-) is formed 
with which the next iteration starts (case(c)). In the nondegenerate case, 
each iteration lowers the residue of the quasiprogram. 

Observe that all the auxiliary problems will be nondegenerate if the aug- 
mented problem (8) is nondegenerate. In this case any two quasiprograms 
of problem (A) obtained in the process of its solution have different residues. 

A one-to-one correspondence exists between each quasiprogram of prob- 
lem (A) and some support program of the augmented problem (8); the value 
of the linear form (4.8) on this program is equal to the residue of the quasi- 
program. In the process of solution we thus move over different support 
programs of problem (8). 

Hence, the problem (A) is investigated by the Hungarian method ina 
finite number of iterations. This conclusion also applies to the degenerate 
case, provided the auxiliary problems are solved using the rules guaran- 
teeing that cycling will not occur (Chapter 4, § 6). 

Thus, a linear-programming problem with bilateral restraints is solved 
by the Hungarian rnethod in a finite number of iterations, each of which is 
made up'‘of several simplex iterations. 


§ 5. The primal-dual algorithm 


5-1. As we have seen, to each iteration of the Hungarian method cor- 
responds some feasible program Y of the dual problem (A). The program 
Y, in turn, defines the auxiliary problem (C,) of the current iteration. The 
solution of the auxiliary problem gives a quasiprogram X of the primal 
problem (A). The optimality criterion, which is in fact the basis of the 
method, enables us to test whether the quasiprogram X is a solution of 
problem (A). The unsolvability test gives a sufficient condition for problem 
(A) to have no programs, If quasiprogram X is not the optimal program 
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and there is no reason to suppose that the problem is unsolvable, elementary 
transformation defined by the solution of the two auxiliary problems (C,) 

and (C,) is used to pass to a new program Y’ of the dual problem (A). The 
next, successive iteration then starts with program Y’ and proceeds as 
before. 

In each step of the Hungarian method the auxiliary problem (C,) has to be 
solved. In general, problem (C,) can be solved either by the simplex 
method or by the dual simplex method. In each of the two methods the solu- 
tion can be obtained either by the first or the second algorithm. Certain 
considerations, however, make it advisable to solve the auxiliary problem 
by the second algorithm of the simplex method. The simplex method is 
chosen because determination of the first program of each auxiliary prob- 
lem does not involve special computations. The basis of the first program 
of the first auxiliary problem may be taken as the set of unit vectors 
€,,.+-,@. The basis components of the first support program of problem 
(Cy) then coincide with the corresponding components 3b; of the constraint 
vector B (b; can always be taken as nonnegative). The first program of each 
successive auxiliary problem may be taken as the solution of the preceding 
auxiliary problem. 

To proceed with the successive iteration in the Hungarian method, we 
require not only the solution of auxiliary problem (C,), but also the optimal 
program of its dual problem (C,). The auxiliary problem is, therefore, 
best solved by the second simplex algorithm, which, besides the optimal 
program of the primal problem, yields also the solution of the dual prob- 
lem. The lowest row of the last principal tableau of the second algorithm 
gives the components of the optimal program of the dual problem. 

Application of the first algorithm is complicated by the fact that to form 
the successive auxiliary problem we must resolve all its restraint vectors 
not entering the preceding auxiliary problem in terms of basis vectors. 
This is unnecessary when using the second algorithm. 

From the above it is obvious that the computational procedure of the 
Hungarian method should be based on the second simplex algorithm. 

Solution of a problem by the Hungarian method thus involves successive 
filling in of several principal tableaus and one auxiliary tableau. 

We will now describe the structure of the auxiliary tableau (Table 7.2). 

In the upper part of the auxiliary tableau (the first m+1 rows) we write 
the components of the augmented restraint vectors of primal problem (A). 
When solving each auxiliary problem (Cy), we use only those columns A, in 
Table 7.2 for which /€F£,. It should be remembered that in auxiliary prob- 
lems the coefficients of x; in the linear form are all zero, 

When solving auxiliary problems, the reader is advised to follow the 
following rule. Any unit vector e, eliminated from the basis in one of the 
steps should not be included in successive bases. This restriction may, 
sometimes, yield a somewhat higher minimum of the linear form of an 
individual auxiliary problem. In other words, this restriction can lead to 
quasiprograms with exaggerated residue. Nevertheless, in general, this 
restriction reduces the laboriousness of computations, since it eliminates 
the necessity of examining the unit vectors e,, e,, ..., ¢, in the process of 
solution. The unit vectors e,, ..., e, entering the system of restraint vec- 
tors of any auxiliary problem should, therefore, not be included in Table 
7.2. We leave it to the reader to show that this restriction in the choice 
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TABLE 7.2 
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of vectors introduced into the basis does not affect the convergence in the 
Hungarian method (see Exercise 5). 

In every iteration, the rows A, 6, 8’, ..., 8*, 6 are filled in the lower part 
of the auxiliary tableau. The part of the auxiliary tableau comprising the 
rows A, 6, 6’, ..., 6*, § will be called the auxiliary subtableau corres- 
ponding to the given iteration. 

In the first iteration of the Hungarian method, the elements of row A 
are computed from the formula 


Ay= 2 aie) J=\1, 2, er) n, (5.1) 
where Y=(y,,..., ¥,) is the first program of the dual problem (A). In subse- 


quent iterations the elements of row A are computed using recurrence 
formula (1.23). The positions / corresponding to zero elements of row A 
(marked by X) give the set £, of the indices of the vectors A, entering the 
system of restraint vectors of the auxiliary problem (C,). The rows 4, 

6’, ..., 6* give evaluations of the restraint vectors A, (J€ Ey) of the auxiliary 
problem (C,) relative to the corresponding basis, 


s= Dap? for je ky (5.2) 


(the coefficients of x, in the linear form of the auxiliary problem are all 


zero), Here the parameters p/}”) are evaluations of the restraints of the 


auxiliary problem relative to its program (&, E&”). In the process of solu- 


tion of the auxiliary problem the elements of) of rows & are computed for 
JE £, only. The remaining positions of 6” are crossed out. The only ex- 
ception is the row &* corresponding to the optimal program of the auxiliary 
problem. Theelements 8} are computed from formulas (1.22) for all 
DiS), 2} xen fh): 

In the last row of the auxiliary subtableau corresponding to each itera- 
tion only those positions are filled which correspond to / for which 67>0. 
The elements of the row § are computed as ratios of the corresponding 
elements of the rows A and 6*. The least element 8, of row § is taken as 
the parameter of the elementary transformation of program Y of problem 
(A) into program Y’, 

In the auxiliary tableau given in this article it is assumed, to be specific, 
that in the first iteration only A, and A, vanish in row A, whereas in the 
second iteration A,=0 also. In row @ cells corresponding to nonpositive 8; 
are crossed out, 

The elements of the subtableaus are denoted by three indices: the num- 
ber of the iteration using the Hungarian method, the number of the iteration 
in the solution of the auxiliary problem, and the number of the vector. To 
avoid complicated notations, we omit indices whenever no confusion, can 
arise. 

As we see, the structure of the auxiliary tableau using the Hungarian 
method is somewhat different from the structure of the corresponding 
tableau using the second simplex algorithm. In each iteration of the simplex 
method one row is added to the auxiliary tableau, whereas in each iteration 
of the Hungarian method a subtableau, consisting usually of several rows, 
is added to the auxiliary tableau. 

The principal tableaus in the Hungarian method (see Table 7.3) are con- 
structed in the same way as in the second simplex algorithm. 
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The row A of the auxiliary tableau corresponding to the first program Y 
of the dual problem (A) defines the set of indices £, and enables us to form 
the first auxiliary problem (Cy). 

Problem (C,) is solved following the usual rules of the second simplex 
algorithm. Principal tableaus are filled successively, and after each step 
in the process of solution of problem (C,) we add a row & to the auxiliary 
subtableau /. Entries are, obviously, written only in those positions 8)’ 
for which /€ Ey. 

The main part of the first principal tableau is filled without any special 
computations. The unit vectors e,, e,, ...,e, appearing in the system of 
restraint vectors of the first auxiliary problem should be taken as its first 
basis. Therefore 


cum| pelea ey fe re te eee 


1for i=, 
e;,= 5, i=}, 2, coe, MH, 
m m 
Cati,e— ry Cie= Py b,, 
m 
Cntr, == 2 Ge y= 1, j=l, 2, eee, mm. 


Here, 6,=0,=...=0,=1 are the coefficients of e,, ¢,, ..., e, in the linear 
form of the auxiliary problem. 

Columns A, and 6 in the principal zeroeth tableau are filled according to 
the general rules of the second simplex algorithm; this algorithm is also 
used to pass to successive principal tableaus. (The reader should not con- 
fuse the notations 6 and 6, of the last column in the principal tableau and of 
its least element, respectively, with analogous notations used to construct 
the auxiliary tableau.) In column o of the principal tableau we write the 
coefficients of the basis variables of the linear form for the corresponding 
iteration of the auxiliary problem. The coefficients o, corresponding to 
the vectors e, are all equal to unity. The coefficients corresponding to the 
vectors A, (J/€£,) appearing in the basis of the support program of problem 
(C,) are all zero. 

The vector A, to be introduced into the basis is determined by the maxi- 
mum positive elements of row $ of the auxiliary tableau. The maximum 
positive 6 (and not the least negative) is chosen because the auxiliary prob- 
lem is to minimize the linear form and not to maximize it. 

The reader is now advised to reread Chapter 5, §5 where the process 
of solution by the second simplex algorithm, using the terminology and the 
notations of the auxiliary problem, is described. 

After several simplex iterations we have the row 6 in the auxiliary 
tableau which contains nonpositive entries only (this refers only to those 
positions in the row which correspond to the restraint vectors of the auxi- 
liary problem). In Table 7.2 this row is marked as &. 

The elements e, in column e, of the last principal tableau, corresponding 
to the basis vectors A, (s,;€ Ey), define a quasiprogram X of problem (A). If 
e419 (the residue of quasiprogram X) in the lower left corner of the tableau 
is zero, quasiprogram X is the optimal program of problem (A), The pro- 
cess of solution then terminates (case (a)). 

Now let e,,,#0. In this case, using formula (1.22) we fill in the empty 
cells in row 6. If all the new entries 6; are nonpositive then, according 
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to the unsolvability test (case (b)) the process of solution is terminated by 
establishing inconsistency of restraints of problem (A). If some of the 8; 
are positive (case(c)), we proceed with the next iteration. To this end we 
fill in the cells of row 6 corresponding to 6; >0 appearing in the ratios 

A ,/6; and select the least element 6,. Further, using the relationship 


Y’ = Y—0,M° 


we compute the successive program of problem A, which is used as the 
point of departure in the next iteration. However, there is no need to 
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FIGURE 17.5 


compute a program of problem (A) at every step. It is better to compute 
the first row of the subtableau pertaining to the successive iteration in the 
Hungarian method using the formula 


A’ =A (0,) =A—0,6°. (5.3) 


The zero entries of the row A’ define a new set Ey and thus a successive 
auxiliary problem (Cy). 

The process is then resumed following the same rules as in the first 
iteration. 
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It should be emphasized that the main part of the first principal tableau 
in each successive iteration is taken as the main part of the last principal 
tableau in the preceding iteration. 

The iterations are continued until either case (a) or case (b) is obtained. 
In case (a) the problem is solved, and in case (b) the problem is shown to 
be unsolvable. 

One more remark. The solution of each auxiliary problem does not re- 
quire, as a rule, numerous iterations. This is so because the solution of 
the preceding auxiliary problem (Cy), taken as the first support program of 
the next problem (Cy) is generally fairly close to the optimal program of 
problem (Cy). In solving the first auxiliary problem we start with a unit 
basis. Computation of the optimal program of the first auxiliary problem 
is not particularly laborious if the number of elements in the set F, is far 
less than m._ If the set E, contains more elements (of the order of m) the 
solution of the auxiliary problem involves rather cumbersome computations. 
In this case, however, far less iterations are required to solve the primal 
problem by the Hungarian method. 

Figure 7.5 shows a block diagram of the solution of a linear-programming 
problem by the Hungarian method. 


5-2, We now illustrate the application of the algorithm by the following example, 
Maximize the linear form 
L (X) =x, +%,+%, +3x,+%;+3x, +x, 
subject to the conditions 
x, + 2x, + 2x, + 3x, +2x, + 4x, +-2x, =5, 
2x, + Xp +4x,— Xt xy+2x,—3x,=7, 
X,+3x,+ x, +4x%,— x,+3x,+2x,=5, 


—X,+2x,4+ X,+ ¥,+2x,— Xet+ x, =2, 
xj20,  f=1,2,...,7. 


All the coefficients of the first restraint are positive, According to Chapter 6, 7-2, the components of 
the first program of the dual problem can be determined as follows; 


1 1 2 3 | 38 1 
A=MXIT> 9° FF OT =1, ”=K4=y¥, 70. 


We form an auxiliary tableau, In row A we write 
a 
A= 2 4; Yi — Cp jot, 2, eens 7. 
=1 


Two elements in row A are zero (A, and A,), Hence, auxiliary problem (Cy) contains, besides the unit 
restraint vectors @ (¢=1, ..., 4), two vectors only, A, and A,. We fill in the main part of the first prin- 
cipal tableau. In the last row, in positions @¢, ..., @ we write the relative evaluations of restraints of 
problem (Cy). We have 


p=, f=1, 2, 3, 4. 


We then compute the relative evaluations of the restraint vectors of problem (Cy). We have 


6= 2 Qj f4; = 3; 6,=7. 
=1 

The evaluations of vectors A, and A, are positive. A, with the higher evaluation is introduced into the 
basis of problem (Cy), The vector A, is resolved in the vectors of the first basis and the coefficients (coin- 
ciding with the components of A,) are written in column Ay, of the principal tableau. The (m-+1)-th 
(fifth) entry in column A, is 6,=7. Then, for nonnegative entries of column A, we compute the compo- 
nents of column @, i.e, , the ratios of the corresponding elements in columns é, and Ag. The last entry 
0,=*/, in column @ is obtained on the basis vector e,. The vector e, is to be eliminated from the basis. 
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TABLE 7,5 (1-4) 
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Resuming the process of solution of the auxiliary problem according to the rules of the second simplex 


algorithm, we obtain after two iterations the solutions of problem (Cy) and (Cy): 


1) 1 16 
s e pa —- — 
(2 ,E \=(5 r 3 > 3 9 0, 0, a) 


4 8 
e — Sciam nae ==» 
M _— (. 9 ’ 9 ? 1) e 
The residue of the corresponding quasiprogram is 17/3, Hence, this quasiprogram is not the solution of the 


problem, 
Filling in the unoccupied cells in row 6° of the auxiliary tableau, we obtain 


es. = — — — —_ — 
é (0, 9 : 9 2 0, 3 : 9 . 9 ) : 


In the last row 8 of the subtableau corresponding to the first iteration we write four elements only (in cells 
where 8, >0). The least element @, in row @ is 3/16, 

We now proceed with the second iteration, The elements of the first row of the subtableau corresponding 
to the second iteration are computed from the formula 


A, =A ,—%9)- 
Row A of subtableau 2 contains three null entries, A\, Ai, Ai. Hence, besides the unit vectors e;(i=1, 2, 
3, 4), the new auxiliary problem (Cy,) contains the restraint vectors A,, A,, and A,. 

As the main part of the first principal tableau in the solution of problem (Cy,) we take the last (third) 
principal tableau in the solution of problem (Cy), The second iteration of the Hungarian method contains 
one step only. The solutions of the third and the fourth auxiliary problems (Cy,,} and (Cy,,,) are also 
obtained in one step each, The quasiprogram 


X* =(1, 1, 1, 0, 0, 0, 0) 


corresponding to problem (Cy,.,) has a zeroresidue, Quasiprogram X® is thus the optimal program of the problem. 
The entire process of solution is represented in Table 7.5 (principal tableaus) and in Table 7,4 (auxiliary 
tableau), 


5-3. Application of the simplex method presupposes knowledge of some 
support program of primal problem (A). When solving a linear-program- 
ming problem by the dual simplex method we start with some support pro- 
gram of the dual problem (A). Computation of a support program of the 
primal or the dual problem is in general no less tedious than determination 
of the optimal program from a given support program. 

The Hungarian method, on the other hand, starts with any feasible (not 
necessarily support) program of the dual problem (A). This is a consider- 
able asset, since in many cases the choice of a feasible nonsupport program 
of the dual problem does not require any special computations. If, however, 
the structure of the problem does not indicate a first feasible program, we 
must apply the method discussed in Chapter 6, 7-6. According to this 
method, problem (A) is supplemented by 


a 


j= 
or 


A 
2 x;=M, x, = 0. 


/=0 


Here M is assumed sufficiently large. The resulting linear-programming 
problem is called problem (M). Its dual, problem (M) is to find the mini- 
mum of the linear form 


p> by; + My, 
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whose variables are subject to the conditions 


m 
2A > ep JH, 2x eee y H, 
y= 9. 


As a feasible program of problem (M) we can, obviously, take the vector 


Y=(y,, 0, eee, 0), 
where 


= max {c,, O}. 
a jee? 


We now solve problem (M) by the Hungarian method starting with program 
Y. In the process of solution the number M is assumed greater than any 
other number with which it is compared. Let 


X* = (x9, Ky 000, Xn) 


be the solution of problem (M). Two cases are possible: 

1. The index /=0 belongs to the set Ey(s), where Y® is the program of 
the dual problem corresponding to the last iteration of the Hungarian method, 
i.e., Af) =0. 

2. The index /=0 does not belong to the set Ey(s), i.e., A®>0. 

In the first tase, the vector 


X* = (Xt) Kay vey Xn) 


is the optimal program of problem (A). In the second case problem (A) is 
unsolvable. Follawing the considerations given in Chapter 6, 7-6, the 
reader can prove this proposition (Exercise 9). Further, it can easily be 
shown that if the process of solution of problem (M) terminates in case (b) 
(problem (M) is unsolvable), the restraints of the primal problem are in- 
consistent (Exercise 10). 

5-4. Let us now consider the special features of the Hungarian (or the 
primal-dual) algorithm in application to linear-programming problems 
with bilateral restraints. 

The bilateral restraints force us to introduce certain modifications in 
the structure of the principal and the auxiliary tableaus. As with a prob- 
lem in canonical form, each iteration of the Hungarian method involves 
filling several principal tableaus and a subtableau of the auxiliary tableau. 

In the principal tableaus and the subtableau of the auxiliary tableau cor- 
responding to an individual iteration of the Hungarian method, we give the 
process of solution of the successive auxiliary problem according to the 
second simplex algorithm. Fach subtableau, -moreover, contains the rows 
A, (a, B), 6%, and §. Inrows A, (a, 6), 6*=6® and (a, B)® (s is the number of the 
last simplex iteration in the solution of the auxiliary problem) all the ele- 
ments are written (the cells in row A corresponding to A,=0 are marked 
by crosses). In row 6 only those elements are written for which the cor- 
responding elements in rows A and 6&* have like signs. 

The elements of row A inthe first subtableau are computed from 


4,= 2 2 Yi—Cp 


where y, are the kernel components of a program of the dual problem (A). 
The elements of row A, of the next subtableau are computed from the 
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recurrence formula 
A; = A,—0,8). 


Row A makes it possible to define the set Fy and thus to determine the 
successive auxiliary problem. In row (a, §) following row A we write the 
symbol a when A,;>0 and the symbol B when A4-<0. The elements in row 
(a, B) for /EZ, are computed following the rules for the solution of linear- 
programming problems with bilateral restraints according to the second 
simplex algorithm (see Chapter 5, 7-3). 

In subsequent rows of the subtableau (6, (a, 6)’, 6’, (a, B)”, ..., (a, B)'"", 68°") 
only entries which correspond to the restraint vectors of the problem, are 
made. In row a, f) the elements for /E€f, are written following the same 
rules as in other iterations with problem (C,). The elements for /€F, give 
the same limits of problem variables as those indicated in row (a, B). The 
elements in row (a, 6) contain the values of extrabasis components of the 
corresponding quasiprogram. 

The elements in row 6* are calculated according to 


8; = p> a; Aj. 


For /€E, 8; =8. The last row 6 of the subtableau is used to determine the 
parameter 6, of the elementary transformation of kernel Y into kernel Y’ 
of the successive program of the dual problem. Kernel Y’ specifies the 
transition to the next iteration of the Hungarian method. 

Successive subtableaus of the auxiliary tableau are filled following the 
same rules. The only difference is, as we have already observed, that the 
row A) is no longer computed directly, but from a recurrence formula in 
terms of the parameters of the preceding iteration. 

To complete the description of the auxiliary tableau we note that in the 
upper part column &Y) is added to the left, column Y to the right, and row 
(a/B) above. Column AY) contains the components 


bf = b, — A, 
Pa : 


of the constraint vector of the first auxiliary problem (y, are computed 
from (4.6)). The elements of row Y are the kernel components of the first 
program of the dual problem. The elements in the upper row (a/B) give the 
upper and the lower limits of variation of the problem variables. 

All the special features of principal tableaus connected with the solution 
of auxiliary problems (each of these is a problem with bilateral restraints) 
are discussed in Chapter 5, 7-3. Here we note a certain inconsistency in 
the notations of Chapters 5 and 7, which need not cause any confusion, In 
Chapter 5 the relative evaluations of problem restraints are denoted by 4,. 
In Chapter 7 the preliminary evaluations of the restraints of the auxiliary 
problem are denoted by p;, and the components of the solution of problem 
(C,)), the dual of problem (C,), are denoted by yj. 

The principal tableaus contain column 6 whose elements are computed 
from formula (7.2) of Chapter 5. The elements of row @ in subtableaus of 
the auxiliary tableau are defined by (4.24) in this chapter. 

When filling the principal tableaus, the following should be observed. 
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Column e, of the principal tableaus contains the coefficients of the 
expansion of the vector 


BO Pe Ue P= ene ee 2 (5.4) 
itly iely 

in terms of basis vectors of the support program of problem (C,). Here the 
parameters y, are computed from (4.6), /y is the set of indices of vectors A, en- 
tering the basis of the support program of the corresponding auxiliary problem. 

When solving the auxiliary problem, the elements of the main parts of 
the principal tableaus are transformed according to formula (7.7) of Chap- 
ter 5. The solution of problem (C,) is used as the first program of prob- 
lem (Cy-). Therefore, the same recurrence formulas are used to transform 
the main part of the principal tableaus when passing to a successive itera- 
tion of the Hungarian method. By virtue of (5.4), this applies also to the 
transformation of column e,. 


We shall illustrate the application of the Hungarian method to linear- programming problems with bi- 
lateral restraints by an example which has been previously considered in connection with the simplex method. 
Maximize the linear form 


L (X) =x, $+ 2x, + 3x, 4% + 2x5 + 3x, + X,— gy — 9 — X19 — 
subject to the conditions 


Nyt Xp t2ey~4+3x,+ 2x,+3r,4+ x,4+%, =7, 
2x, +3x,+ x, +2x+ 3x,+2x,+2x, +X, = 8, 
X, + 2x, + 3x, +2x,+0.5%,+ x4 x, + X19 = 6, 
2x,+ %2+3x,+ x%4+ 2x,+3x,+ +, +%,,=7, 


Ocx<l, j=l, 2...,7, 
xj;=0, /=8, 9, 10, 11. 


The process of solution is given in Table 77 (principal tableaus) and in Table 7,6 (auxiliary tableau). As the 
first feasible program of the dual problem we take the vector Y =(0.8; 0; 0.6; 0). 

In row A of subtableau 1 the parameters A, and A, are zero, Hence, Ey={2,6}. The process of 
solution of the first auxiliary problem is written in principal tableaus 1,1‘, and 1" (see Table 7,7) and in sub- 
tableau 1 of the auxiliary tableau (see Table 7,6). In row (a, B) of subtableau 1 the elements corresponding 
to restraint vectors A, and A, carry the symbol a, since in the first program of auxiliary problem (Cy) the 
variables x, and x, are equal to zero, i.e, , the limit on the left of the interval of definition. Other ele- 
ments of row (a, B) are written according to the sign of the corresponding A, The entry for A,, where 
A,==—0.1, is the symbol §, and the entry for A,, where A,=0,4, is the symbol a. 

The components of quasiprogram X are listed in row (a, B)” of subtableau 1 and in column e, of prin- 
cipal tableau 1". The parameter @, of the elementary transformation of kernel Y into kemel Y’ of a program 
of the dual problem is obtained on j/=3. 

The elements of row A of subtableau 2 are computed from recurrence formula (5.3). The set Ey,, 
contains the indices j=2 and 3, The solution of problem (Cy) is taken as the first program of problem (Cy). 
The main part of principal tableau 2 is, therefore, computed from the main part of principal tableau 1" 
according to the same recurrence formulas as those used to determine the elements ej, in tableau 1" from the 
elements of tableau 1', 

The basis components of the optimal program of the problem are written in column e, of principal 
tableau 7, The extrabasis components of the solution are determined from row (a, B) of subtableau 7 of the 
auxiliary tableau, 


§6. The Hungarian method and bilateral evaluations 
6-1. When solving problem (A) by the Hungarian method we construct a 


sequence of programs of the dual problem (A) which monotonically converge 
to its solution. 
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The value of the linear form of the dual problem on these programs mo- 
notonically decreases approaching the value L(X*) of the linear form on the 
optimal program X* of the primal problem (A). After each iteration we can, 
therefore, easily obtain an upper bound for the optimal value of the linear 
form L(X) of the primal problem (A). 

In some problems each quasiprogram can be used to construct a feasible 
program of the primal problem on which the value of the linear form does 
not exceed its value on the quasiprogram,. In these problems after each 
iteration we can obtain a program and evaluate the difference between the 
value of the linear form on this program and its optimal value. 

In particular, problems for which a feasible program can easily be de- 
termined for any constraint vector B=0 and where the linear-form coeffi- 
cients are nonnegative have this property. This can easily be verified from 
the following considerations. If xX’ is a quasiprogram of the problem, then 
AX'«B. Given the structure of the restraint matrix A we can easily find 
a nonnegative vector xX" satisfying the system of equations 


-AX” = B—AX’, 


The vector X=X’ +X” is a feasible program of the primal problem and, 
since the linear-form coefficients are nonnegative, 


L(X) SL (X’), 


Examples of problems which have the above property are problems whose 
matrices contain a complete system of unit vectors and the transportation 
problem, 

6-2. We now consider the revised Hungarian method (which can con- 
veniently be called the method of bilateral evaluations) which, 
in general, involves covering feasible programs of the primal problem, 

Each iteration of this revised Hungarian method is associated with a 
certain evaluation of the deviation of the program found from the optimum. 
The problem is considered solved when the evaluation falls within a pre- 
determined range. The revised Hungarian method is often preferred in the 
solution of various practical problems when a solution ensuring a given 
deviation from the optimal value of the linear form is acceptable. It is 
noteworthy, however, that to solve linear-programming problems by the 
revised Hungarian method we should know, besides the first program Y of 
the dual problem (A), also a first support program X of the primal prob- 
lem (A). 

We outline briefly the general procedure of the revised Hungarian method. 

We start with a support program X=(x,, x,,...,x,) of problem (A) and a 
feasible program Y=(y,, yx «++ Yn) Of dual problem (A), Problem (A) is given 
in canonical form: 

Maximize the linear form 


LiX= Dew, (6.1) 
subject to the conditions 
a 
Pa tuts= bp i=l, 2, ..., m, (6.2) 
x20, j=l, 2, ..., 4. (6.3) 
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Further, let y,, ys. «++» Ja be an arbitrary given set of numbers, Multi- 
plying the i-th restraint in (6.2) by y,, summing the right- and the left- 
hand sides of the resulting equalities, and subtracting (6.1) from the sum, 
we obtain 


Zoin—b (X) = 2 2 (2; ;Y;— 6) %. 


Setting 
A,= Layi—ep (6.4) 
we find 
n m 
2 Apy= bin —L (m). (6.5) 


Thus, the problem of maximizing L(X) under conditions (6.2) and (6.3) 


a 
can be stated as a problem of minimizing p> A,x, under the same conditions 
=1 


and with constant y,(i=1, 2, ..., m). 
Now let Y=(y,, ..-, J,) be a feasible program of problem (A), the dual of 
problem (6.1)—(6.3). Then 
A,>0, j=1, 2, ..., 4. (6.6) 


If X is a program of problem (A), the value of linear form (6.5) can be used 
to evaluate the deviation of program xX from the optimum. Indeed, 


> CX; & L (Xopt) <= p> b,Y;. 


fz 


The process of solution can, therefore, be terminated when the difference 
m > n 
b.y,— 2)¢,%,= QA-x, 
2 iVi ae p> i*j 


is comparable to the error inherent in measurements of the initial data of 
the problem, 

Let, as before, £, be the set of indices j/ such that A,=0, where 4, is 
computed from (6.4) starting with a program Y of the dual problem. Let 
Ey be the set of indices of the basis vectors of the support program X. 
Moreover, we introduce the set Ey , of allthe indices / which belong to Ey 
or Ey (Ex, y = Ex UE,). 

The pair of programs X and Y is associated with an auxiliary problem 
(Dx, y) which is to minimize of the linear form 


Ae 
Aes fy (6.7) 
subject to the conditions 
A,x,=B, 
oP y yy (6.8) 
x,20, JEEx, y. (6.9) 


We solve problem (Dx, y) by the second simplex algorithm. As the first 
program of problem (Dx, y) we naturally take the given support program X, 
Clearly, every program of problem (Dx, y) is also a program of problem (A). 
The components of program X of problem (A) whose indices are not in Ey y 
are zero. 
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Let X* be the optimal program of problem (Dx, y). Relationship (6.5) 
shows that the value of linear form (6.7) on program X* can be called the 
residue of program X* of problem (A), 


* A,x°, 6.10 
° eBoy my ( ’ 


The optimality test for programs of problem (A) is stated as fol- 
lows: 

Program X* is the solution of problem (A) if its residue 
is zero. 

As in the Hungarian method, we introduce the parameters 


87= 2 aii, GaN Died it (6.11) 


where M*=(u1, py, .-., Um) is the solution of problem (bx, y), the dual of auxi- 
liary problem (Dx, y). The solutions of the auxiliary problem (Dx, y) and of 
its dual problem (Dx, y) define a new support program X’ = X* of problem (A) 
and a feasible program Y’ of problem (A). The residue of program X’ is less 
than the residue of the first program X. 

Program Y is transformed to program Y’ of problem (A) by means of the 
elementary transformation 

Y’ = Y—6,M*. (6.12) 


The parameter 6, of the elementary transformation is computed from 


A 
6, = min 2. (6.13) 
b;>0 1 
The parameters A; of one iteration are related with the parameters A, of 
the preceding iteration by the recurrence formulas 


A; = A,—0,8;° (6.14) 


Support program X’=X* of problem (A) and feasible program Y’ of prob- 
lem (A) define a new auxiliary problem (Dx,y). As the first program of 
problem (D x,y) we can, clearly, take the optimal program X* of problem 
(Dx. y). 

This process is repeated until a program with zero residue is obtained. 
Problem (A) is always solvable, since by assumption the process of solu- 
tion starts with a given support program X of problem (A) and a given 
feasible program Y of problem (A). 

6-3. The algorithm of the revised Hungarian method (the method of bi- 
lateral evaluations) is different from the computational procedure of the 
ordinary Hungarian method. 

In the Hungarian method, the relative evaluations 6, of the restraint vec- 
tors A, of the auxiliary problem are defined as 


8” = 2 a hy, JEE, 


(the coefficients of x, in the linear form of problem (Cy) are all zero). 
The linear-form coefficients of problem (Dx, y) are equal to A,. There- 
fore in the revised Hungarian method 


P= Bapr—Ay JEEx.y. (6.15) 
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Here ! is the number of the last iteration in the solution of the auxiliary 
problem. 
In the Hungarian method 
8) = 6" JEEB£,, 
and in the revised Hungarian method 
=O +A, JEEx, v. (6.16) 


With j€Z,, we have A,=0 and 6;=6)9. With /€Ex-, we have 6}’=0 (opti- 
mality test of program X of problem (Dx, y) computed by the simplex method). 
Applying (6.16), we find 

6;=A, for jE€Exs, 


and, consequently (see (6.1 4)), 
A;=A,(1—6,), if JE Exe. (6.17) 


As the first program of problem (Dx, y-) we take the optimal program of 
problem (Dx y). Therefore, Ey-=E£x- and relationship (6.17) holds for all 
JEEx:. 

Some of the A,(/€ Ex y) are positive (otherwise, the residue e*=0), Let, 
for instance, A,>0. Further, since all Aj>0, we have 


Ai, 
(1 —6,) ey Vi 0. 


Clearly, the optimal program of the auxiliary problem will not change 
if all its linear form coefficients are multiplied by a constant positive num- 
ber. Formula (6.17) and the relationship Ey-=Ex. therefore enable us to 
replace the solution of problem (Dx-, y-) with the solution of the reduced 
auxiliary problem with linear form 


A,x 
ie ve 


and the restraints of problem Ey y, 

If A, is a restraint vector of the two auxiliary problems above, the 
linear-form coefficients of the two problems corresponding to the vector 4A, 
are equal. This makes it possible to do without various auxiliary operations 
in each iteration of the revised Hungarian method and to effect the transi- 
tion from the auxiliary problem to the reduced problem, 

The value of the linear form of the auxiliary problem on every support 
program X differs from the value of the linear: form of the corresponding 
reduced problem on the same program by the factor 


v,=(1— 60?) (1-00)... (1-80), 


where I! is the number of the iteration of the revised Hungarian method, The 
residue of program X™ is, therefore, equal to 


ey = Ve (6.18) 


where yp," is the value of the linear form of the reduced auxiliary problem 
on program xX", .¢; is conveniently written in the left bottom corner of the 
principal tableau containing the solution of the /-th reduced auxiliary 
problem, 

The preceding remarks are sufficient to construct a computational 


) 
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procedure of the revised Hungarian method on the basis of the primal-dual 
algorithm of the Hungarian method. This is left to the reader (Exercise 11). 

We shall now show that as long as we do not have the solution of problem 
(A), the parameter 6, of the elementary transformation falls in the interval 
(0, 1). In other words, 


0< <1, (6.19) 


if the residue of program X" is positive. 
The inequality 6,’<1 for e;>0 follows from (6.18), On the other hand, 
if some of 6) are positive, 


6,= min *é50, 
b>0 j 


To prove the left-hand side of inequality (6.19), it thus suffices to show that 
some of the parameters 6 are positive. 

If 6;<0 for all j, problem (A) is unsolvable. Indeed, if 6; <0 for all j, 
the vector Y(6)= Y—0M* is a program of problem (A) for any 6>0 (see (6.14)). 
The value of the linear form of problem (A) on program Y (6) is equal to 


L(y @) =L(y—0m") =L(Y)—0 3 oui =1(r)—-8. Ax; = L(Y)—6e*. 
= x,Y 


When e*>0, an increase of 6 results in unlimited decrease of L(Y’). 
According to Chapter 3, Lemma 1.3 this indicates that problem (A) has no 
feasible programs. This contradicts the assumption that the process of 
solution by the revised Hungarian method starts with a given support pro- 
gram of problem (A), Thus, some of § are positive. This proves inequa- 
lity (6.19). 

6-4. We shall now show that the revised Hungarian method gives the 
solution of a linear-programming problem after a finite number of itera- 
tions. The proof follows the outline below. We first show that each itera- 
tion of the revised Hungarian method either yields a new support program 
of problem (A) or reduces the number of restraint vectors with positive A, 
in the successive auxiliary problem. The number of support programs, 
and also the number of restraint vectors with positive A,, are finite. A 
method in which a support program has already been examined does not re- 
cur or which reduces the number of vectors with positive A, in each itera- 
tion is a finite method. 

When passing to a successive iteration, two cases may arise in the re- 
vised Hungarian method: the parameter 6, of the elementary transformation 
is obtained either on /eEx,y or on J/EEx,y. We shall consider these two pos- 
sibilities. 

1. The parameter 6, is obtained on /#eEx,y. Then 


Aj=0 (/€ Ey), 


while A,>0. Moreover, we have seen that 6,>0 if X* is not a solutten of 
problem (A). It follows from (6.14) that under these conditions 6;>0. In 
problem (Dx ,y/) the parameter 6, is an evaluation of the restraint vector Aj. 
Hence, some of the restraint vectors of problem (D x,y’) have positive 
evaluations relative to the basis of program xX’. In the nondegenerate case 
(nondegeneracy of problem (Dx. y-) which follows from nondegeneracy of 
problem (A)), the first step of the iteration will yield a program of problem 
(A) with a lower residue e*. Program Y’ is not affected in the process of 
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solution of the auxiliary problem (Dx,y-). The decrease of the value of the 
linear form of the auxiliary problem is, therefore, due to an increase of 
the linear form of the primal problem. 

If jeEx,y the transition to the next successive iteration in the revised 
Hungarian method involves motion over different support programs of the 
problem, Using rules which prevent cycling we can extend the result to 
the degenerate case also. 

2, The parameter 6, is obtained on /E€Fx,y. We shall prove that in this 
case the transition to the next successive iteration of the revised Hungarian 
method reduces the number of basis vectors of the auxiliary problem whose 
A, are positive. This proposition is obvious if the number of basis vectors 
with positive A, is reduced in the process of solution of problem (Dx,y). In 


this case 
Ex DEx- = Ex:. 


Now let Ex and 2x»=E£x contain the same number of indices of restraint 
vectors with A,>0. The index / on which 6, is obtained belongs, by as- 
sumption, to the set &x,y. From (6.19), 


0, = Ay So. 
oF 


Hence A,>0. In other words, /€ Ex =Ex:. 


Thus, on the one hand, A;=6 (@, is obtained on /). On the other hand, 
JE Ex =Exe. From equality (6.17) and the right-hand side of inequality (6.19) 
we see that these relationships are consistent only if the residue of program 
X* — X’ is zero (here X’ is regarded as the solution of problem (Dx:,y’)). 

Thus, if the solution ofthe problem has not yet been found, the parameter 
§, of the elementary transformation can be obtained on /€Ex y only if 


Px DExe = Ex, 


This proves that there are a finite number of steps in the revised Hun- 
garian method, 


6-5, We illustrate the application of the revised Hungarian method by the following example, 
Example 1. Maximize the linear form 


L (X)= 13x, + 10x + 9x, + 6x4 -+ 17x, + 14x, + 15x, + 14.52%, + 29x, + 14x,, 
subject to the conditions 


1.5x, + Xg-+ 17x, + 12e, + 18x, + 7.5x, + 5x, + 4x, + 6.5x, + 2.5x,,.—=7, 
Ox, + 13x, + 16x, + 5x, + 4x, +4x,+6x, +7x, +15x, + 8x,, = 14.5, 


x,=>0, j=1, 2, eet 


Solution. The process of solution is givenin Table 7. 9(principal tableaus) and T able 7.8 (auxiliary tableau). 

As the first support program of the problem we take the program with the basis components x, =0.656; 
x, ==0.373, and as the feasible program of the dual problem the vector Y =(2.095; 1,095). In row A of the 
first subtableau of the auxiliary tableau A, and A,, are zero, Therefore, Ey={1; 10}. . The basis of the 
first support programs consists of vectors A, and A,. Hence, Ey={2, 3} and Ey y={I1, 2, 3, 10}. 

The linear form of the first auxiliary problem is equal to 


A,x, + A,X, =6.333x, ++ 44, 1 4x,, 


and the restraint vectors of problem (Dy y) are A,, Ag, Ay, and Ay. The simplex solution of the problem 

is given in principal tableaus 1 and 1' (see Table 7.9) and in subtableau 1 of the auxiliary tableau (Table 7,8). 
The parameter 6, of the elementary transformation, defined as the least entry in row @ of subtableau 1, 

is equal to 0,157, The elementary transformation of program Y of the dual tableau into program Y’ yields 

Ey, ={9, 10}. The basis of the solution of problem (Dy y) comprises the vectors A, and Ay. These 

vectors also constitute the basis of the first program X’ of problem (Dx: y-). Hence, Ey, =z {3, 10} and 

Ex y/={3, 9, 10}. The residue of program X’ is equal to 9,081, 
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According to 6-3 it is advisable to substitute a reduced auxiliary problem for problem (Dy, y-). The 
restraint vectors of the reduced problem belong to Ey, y,, and its linear form is 


>» A pey= » a 


JeByxs ys JeEx y: 


This eliminates the necessity of making any changes whatsoever in tableau 1' before starting the solution of 
the next auxiliary problem, 
In the first row of subtableau 1, therefore, we write the values 


tableau 


FIGURE 7.6 FIGURE 7,7 


The simplex solution of the auxiliary problem is given in principal tableaus 1' and 2 (see Table 7,9) and 
in subtableau 2 of the auxiliary tableau (see Table 7,8), The basis components of the solution of the reduced 
auxiliary problem are written in column é, of principal tableau 2, The (m--1)-th (third) entry in column e, 
is the value of the linear form of the reduced auxiliary problem on its optimal program, To obtain the linear 
form of problem (Dy, y,) on this program (the residue of the corresponding program of the primal problem), 
we multiply é@g+s,¢ by (1—§,). The residues are written in the (m-+1)-th position of column By (to the 
left Of Cm44,¢)- 

The solution of the reduced auxiliary problem is a support program of the primal problem, Its basis 
components are 

x, =0.0712, x; = 0.891. 


Hence, Ey. = {3, 9}. Elementary transformation with 0, =0.0272 yields feasible program Y” of the dual 
problem, The elements in the upper row A” of the subtableau corresponding to the successive iteration of 
the revised Hungarian method are computed from the parameters of the preceding iteration using the formula 
A, —?0’ 8°’ 
yet as Me 
J 1—6. 


The parameter A)=0 for j=7 and 9, Hence, Ey*={7, 9} and Eye ye={3, 7, 9}. The linear form 
of the reduced auxiliary problem with restraint vectors A,, A,, A, on the optimal program of this problem is 


AY xt +B7 x) +A) xf = 44, 14-0-+0-0.847 +0-0.299 =0. 


The support program X” of the primal problem with basis components x, = 0.847 and x, = 0.299 has a 
zero residue, This program thus solves the problem. 

Figure 7,6 shows the graph of the program residue in successive iterations of the revised Hungarian method, 
This figure also shows the variation of the linear forms of the primal and the dual problems on the corres- 
ponding programs, Successive plotting of these graphs in the process of solution makes it possible to dis- 
continue the computation as soon as the residue is comparable to the error of measurements of the initial data. 
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Example 2 Maximize the linear form 


L = 3x, — x, + 8x,+2x,—x,+9x, 
subject to the conditions 

—6x,+9x, +3x,—2x,—x, < 12, 
—4x,+3x,—3x,+ x,—x, <5, 

2x, + 8x,— 5x, + 6x,— 8x, + 4x, = 20, 
— X%,—3x,— 4x, — 8x, + 4x, 10, 

5x, + X%y-+2x,+ 4x, +9x;, +52, & 24, 

x;=0, fel, 2528.58. 

Solution. This problem has already been considered as an illustration to other linear- programming 
methods. Table 7.11 (principal tableaus) and Table 7, 10 (auxiliary tableau) show the process of its solution by the 
revised Hungarian method. Thesystem of additional unit vectors was taken as the basis of the first support program. 
The first program of the dual prbolem was taken as Y=(0, 0, 0, 5, 14). 

Figure 7.7 shows the graphs of L(X), L(Y), and the program residue in successive iterations, 


6-6. In conclusion of this chapter we wish to mention some advantages 
of the Hungarian method which distinguish it favorably from other linear- 
programming methods, 

To compute the optimal program by the Hungarian method it is not ne- 
cessary to determine first a support program of either the primal or the 
dual problem, The process of solution can start with any feasible program 
of the dual problem, and the solution of a problem of bilateral restraints 
may start with an arbitrary m-dimensional vector Y. 

Given a support program of theprimal problem and a feasible program 
of the dual problem, we may use the revised Hungarian method (the method 
of bilateral evaluations) to obtain, in some cases, an approximate solution 
of the problem with an error not exceeding a predetermined value, after 
appreciably fewer operations than those required to obtain the exact com- 
ponents of the optimal program, 


EXERCISES TO CHAPTER 7 


1, Using the Hungarian method, maximize the linear form 


L=2x,—4x,+%,+%,—2x, 
subject to the conditions 
X,—~— 3x, +2x, —Z; ed I, 
— 2x, +3x,— X,+2,=5, 
2x, X¥y+2x,+51,+2%,=10, 
x;=0, j=1, 2, 3, 4. 


2. Solve the problem in Exercise 1 imposing the additional restraints: x;Q2, j=, 2, 3, 4, 5. 

3, Solve the problem in Exercise 1 by the revised Hungarian method (the method of bilateral evaluations) 
starting with a support program X=(0, 0, 41/29, 31/29, 53/29) and a feasible program of the dual problem 
Y=(0, 0, 3). 

4, Starting with a feasible program Y==(0, 0, 1) of the dual problem, find the parameters a in the 
problem which is to maximize linear form 


subject to the conditions 


—x,+ ar, +7 4,4 ax,=3, 


7 
ate at ax tone? 


3xX,+ 2x,z+ 2x,+ 2,=65, 
xj=-0, j=1, 2, 3, 4, 


for which cases (a), (b), and (c) of the Hungarian method apply. 
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5. Show that the convergence of the Hungarian method is not impaired if the artificia) unit vectors 
eliminated from the basis are not included in successive bases. 

6, Let the vector M° = (p; sei’ B,,) be defined by (4.16). Show that M® is the kernel of the solution 
of the dual problem (4.8), (4,11), (4.12). 

7, Show that formula (4,5) applies to a problem of minimizing linear form (4, 1) subject to conditions 
(4.2), (4.3) if yy are defined as 

={% for A; <0, 

W=\p, for A;>0. 


8. Draw a block-diagram of the primal-dual algorithm of the Hungarian method for problems with bi- 
lateral restraints. 

9, Let problem (M) (see 5-3) be associated with a linear- programming problem, Let X¥ = (xq, X1, 00+, Xn) 
be the solution of problem (M). If j=0 belongs to the set E As) where Y“) is a feasible program 
of the dual problem corresponding to the last iteration of the Hungarian method, then X*°=(x,, ..., Xn) 
is the solution of the primal problem, If j=0 does not belong to Evin the primal problem is unsolvable, 
Prove, 

10, Prove that if the process of solution of problem (M) terminates in case (b) (problem (M) is unsolvable), 
the restraints of the primal problem are inconsistent. 

11, Using the remarks given in 6-3 and the examples described in 6-5, compile a computational pro- 
cedure and draw a block-diagram of the revised Hungarian method (the method of bilateral evaluations). 

12, Give a geometrical interpretation of the revised Hungarian method (the method of bilateral evalua- 
tions) in the (m +-1)-dimensional space. 
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Chapter 8 
FINITE METHODS OF LINEAR PROGRAMMING 


The various methods of linear programming can be divided into two 
classes. The methods of the first class (the so-called finite method) 
ensure a solution of a problem after a finite number of steps. The methods 
of the second class (iterative methods) involve an infinite number of 
iterations and, generally speaking, yield an approximate solution of the 
problem. The quality of the approximation depends to a large extent on the 
number of iterations carried out. Most of the iterative methods are actual- 
ly numerical methods for solving rectangular games stated in terms of 
linear programming. 

In Chapters 4 to 7 we studied in detail the qualitative and computational 
aspects of three principal methods based on essentially different premises. 
These were the simplex method, the dual (simplex) method, and the Hunga- 
rian method. In this chapter we shall consider some problems which, 
since- they can be solved by all the finite methods, were not discussed when 
dealing with each method separately. 

Until now our discussion of the methods and of the corresponding algo- 
rithms was presented mainly with reference to the linear-programming 
problem in canonical form. Uniformity of notations simplifies the analysis 
of the methods and the description of the computational procedures. This, 
however, should not be interpreted to indicate that each applied problem 
must first be reduced to canonical form. Retaining the original form of 
the problem we often greatly reduce the bulkiness of the computations. The 
problem with bilateral restraints given in Chapter 7 serves as a good 
example of this. In §1 the simplex method is extended to linear-program - 
ming problems in arbitrary form. Following the suggestions given in this 
section, the reader will learn to adapt the other finite methods and the 
corresponding algorithms to problems written in arbitrary form. 

In §2 we discuss some means of perfecting the simplex and the dual 
simplex methods which will simplify the corresponding computational pro- 
cedures. Modification of the methods discussed in this section enables us 
touse the memory storage space of computers, which is the bottleneck in all 
modern computing machines, more economically. 

In § 3 the finite methods are classified. The various features outlining 
a unified approach to all the finite methods of linear programming are con- 
sidered. 


§ 1. Finite methods and linear-programming problems in 
arbitrary form 


1-1, In this section we shall deal with the general linear-programming 
problem given in arbitrary form. The restraints of this problem comprise 
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both equalities and inequalities; the nonnegativity restraints are imposed 
only on some of the problem variables. Any of the finite linear-program - 
ming methods can be suited to problems in arbitrary form. This adaptation 
should be made setting out from the general outline of the method described 
for the problem in canonical form and drawing upon the geometrical inter - 
pretation of themethod. We shall deal here with only one method, the 
Simplex procedure. 

Consider the linear-programming problem of maximization of the linear 


form 
n 


Pa, (1.1) 
subject to the conditions 
. =b,, i=l, Parry 1 1.2 
Bowron f=p4+1, p+2,...,m; es 
x, 0, J=94+1, g+2,...,4, (1.4) 


Equations (1.2) should, naturally, be assumed to be linearly independent; 
otherwise some equations could have been omitted. Moreover, we shall 
assume that the system of vectors 


Aj=(Gip Mp eeertg)?, Jl, %eeerg 


is linearly independent. This condition does not detract from the general- 
ity of our arguments: it can always be fulfilled by reducing the number of 
variables which do not satisfy restraint (1.4) (see Chapter 2, 4-1). The 
last assumption ensures the existence of support programs for problem 
(1.1)—(1.4). 
The definition of a support program given in Chapter 2, 4-2 can be re- 
stated with reference to problem (1.1)-—(1.4) as follows. 
A feasible program X=(x,, x,,...,%,) of problem (1.1)—(1.4) is called a 
Support program, if there exists a regular square submatrix 
Qi), Qi if, oo + Biv, 
B15), isfy ++ Qs), 


Ax = 
Biyf, VUygln ++ Us, 
of the matrix of coefficients A=|]@,,|lm,., such that 


(a) i=a fora=1, 2,..., p> 4=f for P=1,2,...,93 
(b) if gq<j and x,>0, then /=/, for some B>q+1; 


(c) aut #5, for a=l, 2, 000, Se 


The rows of the matrix A, are constructed of restraints (1.2) and some 
of the restraints (1.3), all such inequalities being satisfied by program X 
as strict equalities. Among the columns of A x there are all the columns of 
the matrix of coefficients A which correspond to the components x, which 
are either not restrained by the nonnegativity requirement or else are 
positive on program X. The submatrix A, generated by the support program 
X will be called the basis of this program. According to the 
general definition given in Chapter 2, 4-7, the support program X of prob- 
lem (1.1)-(1.4) with basis A, is said to be nondegenerate, if 


x, > 9, p=g+l,....5; 
n 
PT Sead b fei, a=1l1, 2, «way Os 
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Proof of the equivalence of the above definitions of a support program 
and of nondegeneracy of a program of problem (1.1)-(1.4) with the corres- 
ponding definitions given in Chapter 2 is left to the reader (Exercises 1 
and 2). 

1-2. The simplex method will be described here with reference to the 
nondegenerate case — all the support programs of problem(1.1)-(1.4) are 
nondegenerate. 

Let a support program X of problem (1.1)-(1.4) be known and letits basis 
Ay be formed by the elements occupying, in matrix A, the intersections of 


the rows and the columns with the numbers i, i,,...,i, and j,, j,,....,Jj, res- 

pectively. We begin to solve the problem by testing program X for opti- 

mality. To this end, we determine the solution (A, 4,,...,4,)of the system 
Dighe=ly» B=, 2,...48, (1.5) 


If 


A, 0, a=p+1,...,8 (1.6) 
and 
A,=0 for j=1, 2,...,a, (1.7) 


then, according to the optimality criterion for problem (1.1)-(1.4) (Chap- 
ter 3, Theorem 5.2), program X is optimal (case (a)). If, however, atleast 
one of relationships (1.6), (1.7) does not apply, program X is transformed 
by elementary transformation which either establishes unsolvability of the 
problem (case (b)) or yields an improved support program X’ (case (c)). 

We recall the geometrical interpretation of elementary transformation 
in the simplex method (see Chapter 4, 2-2). A support program corres- 
ponds to a vertex of the polyhedral set of the problem. We choose one of 
the hyperplanes passing through this vertex and satisfying the inequality 
restraints and consider the edge of the polyhedral set contained in the inter- 
section of the remaining hyperplanes. The elementary transformation as- 
sociated with the cnosen hyperplane is a translation along the edge indicated. 
If the problem has been reduced to canonical form, elementary transfor- 
mation of its support program is invariably associated with one of the coor- 
dinate hyperplanes. In the general case, elementary transformation may be 
generated either by a hyperplane of the form x,=0 /=q}+1,..., a), or by a 
hyperplane 


n 


Brires= Op i=p+l, p+2,...,m, 


associated with the corresponding restraint in (1.3). Therefore, when 
analyzing elementary transformations of a program of problem (1.1)-(1.4), 
it is advisable to distinguish between two cases. 

1-3, We consider the first case. The elementary transformation is de- 
fined by the coordinate hyperplane x,=0(k #/,B=1, 2,...,s) (elementary 
transformation of the first kind). 

Under this transformation, the support program X is transformed into 


X (8) = X—OH =x{x, (6), x, (0), +, X, (8), 
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where the vector H=m(h,, h,,...,4,) is determined by the relationship 


Xm if j=f B=1, 2,...,5; 
hye —1, if fuk (1.8) 
0 in other cases. 


Here x, are computed from the equations 


2H t= Aye a=1l, 2, coe, Se (1.9) 
It follows from (1.8) and (1.9) that 


n 


Py ries*s OF, for a=1, 2,...,8 

for any 6. . 
We write out the conditions imposed on § under which X(6) is a feasible 

program, i.e., satisfies the remaining restraints in (1.3) and constraints 

(1.4). Since 

x, (0) =~, +6, 


the parameter 6 must be nonnegative. Insofar as the other restraints of 
system (1.4) must also be satisfied, we have 


Xp, — Oxy, = 0, B=¢+1, coe Se 


Thus, for all the components of the vector X(§), beginning with the (¢+1}th, 
to be nonnegative, it is necessary and sufficient that 


.0<06<6;, (1.10) 
where et Xp, 
: X_>o “BR (1.11) 
bs 


If Xpp <0 for B=q+1,...,s the vector X(6) has nonnegative components x, (6) 
(j=q+1,...,2)for any nonnegative 6. In this case we take 6,=o0. 


We define ‘ 
AC ab, — 
i Pe 
$ 
6 = Parise — Aum (1.12) 
$1 5 2 sg 


Applying (1.8) and (1.12), we have 
A‘ (8) =b, — 9 a At!) + 98(0), 
(6) b; Pres 0) + b5{ 
Restraints (1.3), to be satisfied by the vector X(6), are therefore rewrit- 


ten in the form 
AM + 05 > 0, i=p+l,...,m. 


Solving inequalities (1.13) for 60 we obtain (1.13) 
b< 6, (1.14) 
where ; 
A 
6° = min (—4n). 1.15 
. of <o oy ( ) 


If &>.0 for all /, we take 67=oo. 
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Inequalities (1.1) and (1.14) define the set of 6 on which X(6)is a feasible 
program of problem (1,1)—(1.4): 


o<§<6,, 
where — 
0, = min {6,, 6°}, (1.16) 


where 6, and 6° are determined from (1.11) and (1.15), respectively. 
We compute the increase in the value of the linear form (1.1) caused by 
transformation from X to X(6). We have 


2 CX, =% c,x,—O 63 cpt — te) : 


Applying systems (1,5) and (1.9), we have 
8 8 8 & 8 k 
p> “spb = 2 (3 Figd ye) *be = p> (Sa, an) he aa Putighe 
Hence, 
n 
L (X (0)) = D) cy, (6) =L (X)—OA,. (1.17) 
i=! 


Consider the elementary transformation associated with the hyperplane 
a 
Pa tists =i, (¢=p+1, p+2, eee s) 


(elementary transformation of the second kind). 
Under this transformation program X is transformed into 


X (6) = (x, (6), ..., *,, (8)) = X—O6H, 
where the vector H=(h,, h,, ..., &,)is determined from 
h = { ( if i=f, B=1, 2, ..., 8, (1,18) 
4~~\0 in other cases. 


The e, satisfy the equations 
é 1 fora#t, 


Paiste = 1 for a=t. (1.19) 


J.et us find the exact limits of § between which X(6)is a feasible program 


of problem (1.1)-(1.4). From (1.19), 


A 6 n 6 s b, , if a#f, 
Piss = t= i 6 if ant. (1,20) 


Therefore, restraints (1.2), (1.3) are satisfied for i=i,a=1, 2, ..., s,if 
and only if §is nonnegative. 
We define 


rit nr 5 : 
OP = 2 sity 2D *isg°ot b=], 2,-2.4, Mm 
Then 
AW) 8) = 5; — 2 a; x, (8) = AM + 06) , 


Since 6 is nonnegative, applying restraints (1.3) to be satisfied by the vec- 
tor X(6) we obtain 
6<6;, 
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where 


‘ Af“ 
ame (— 3): (1.21) 


If &>0 for all i, then f=oo. 
From the definition of X(6), 


x,()=0 for J/#f, P=1, 2,..., 8 
Hence, restraints (1.4) are fulfilled if and only if 
*, (8) = x), — beg, > 0, B=¢+1, oor, S, 


or, since § is nonnegative, 


6<6,, 
where 
, *j 
O=min —=, (1.22) 
ep, > ° 
"B>e 
If e,,<0 for B=q+1, ..., s,we take 6’=oo. The required limits of 6 are thus 
specified by 
0<6<8,, 


where 
6, = min (0, 0), (1.23) 


where 6 and § are computed from (1.22) and (1.21), respectively. 
Under a given elementary transformation of the second kind, the linear 
form is written as 


L(X(O) =L (0 > chy. 


Applying (1.18), (1.5), and (1.19), successively, we obtain 


Rn t 8 
Pact= Bienen = 2 (3 The ) es 3 (3 414 tu) pa 


Hence 
L(X (6)) =L (X) —6A,. (1.24) 
1-4. We now analyze cases (b) and (c), Let some of the A,(a=p+1, ..., s) 
and A,(/=1, 2, ..., 2) be negative. We choose one of these negative numbers 


(e.g., the one of greatest absolute value). Now the direction of the solu- 
tion process is determined by whether one of the A, or one of the 4, has 
been chosen, We shall deal separately with each of the two cases, 

1. Let A,<0 be chosen. In this case, we apply elementary transforma- 
tion of the first kind defined by hyperplane x,=0. We compute 0, from 
(1.11), (1.15), and (1.16). It is possible that 6,=o0o. This is equivalent to 
the following set of restraints: 


Xgy = 0, p=¢+1, q+ 2, eee, S; 


5 0, t=, 2, ..., m (1.25) 


If §,=00, X(6) is a feasible program of problem (1,1)—(1.4) for any 620. 


But according to (1.17) 
lim L (X (6)) = co 
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Thus, if conditions (1.25) are satisfied (case (b)), the problem in question 
is unsolvable, 
Now assumethat conditions (1.25) are not satisfied (case (c)) and, conse- 
quently, 6,<0o, Let 
X' = X (6,). 


We shall verify that X is a support program of the problem. First let 
6,=6', or equivalently, 


x 
6, = ? Xp, >0, 
rk 


for some r2>q+1. 

Since x,,#0, it follows from Chapter 4, Theorem 2.1 that matrix Ay, 
which is obtained from the basis Ay of program X by replacing the r-th 
column by the column (4; 4, @;,4, +... ie)", 1s regular. 

Program xX’ is thus a support program and its basis is a square matrix 
of order s. 

Let us consider the possibility: 

6, = 6°. 
Here 


AM) ; 
t= tx 8<0 
Rk 


for some/+#i,, a=1, 2, ..., s. Obviously, 
n 


p> a, ,X,= 5; 


x, =9, if j#s, P=1, 2, ..., sand/#k. 
Therefore, if we verify that the matrix 


Giz, Fiz, oes Qi.) ise 

Gish fis: m8. © bit, Qik 
AW ok kaw ee ee ENG 

Bigj, Fish, = ++ Teh, Usk 


Gij, Bij, «++ Gj, Ok 


is regular, we shall have proved that xX’ is a support program. 

The proof will be carried out by reductio ad absurdum, If the columns 
of matrix Ax are linearly dependent, its last column is a linear combina- 
tion of the first s columns since these columns are a priori linearly inde- 
pendent. According to (1.9), the coefficients of the linear combination are 
Xap: However, by assumption, 


s 


Hence, the assumption of linear dependence of the columns of A,-is false; 
matrix Ax is regular, | 

Program X is thus a support program, and its basis Ax is a matrix of 
the order s+1 which is obtained by adding a row (g,,, a,,, ..., @j,,@,) and a 
COlUMN (4; 4, Gig, +++» A ,k,4,)7 to the matrix Ax. 

By assumption X is a nondegenerate support program of problem (1.1)— 
(1.4). 

Hence, x, >0, AM>0, and, consequently, 6,>0, Referring to (1.17), we 
find that 


L(X') =L(X)—6,A,>L (X). 
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2, Let A,<0 be chosen. In this case, the program is improved by 
taking an elementary transformation of the second kind— that generated by 
the hyperplane 

"i 
p> GX ,= by, « 


We compute 6, from (1.21)—(1.23). If 6,=0o or, equivalently, if 
hk, = 0 for a l, eee, S; 
ly Cot B=¢+ s 


B= 3 41; ¢ = Ofori=I, 2, coe, M, (1.26) 
then X(6) is a feasible program of problem (1.1)—(1.4) for any 620. Equality 
(1.26) shows that the problem in question is unsolvable (case (b)). 

If conditions (1.26) are not satisfied, §,<co and the program X’=X(6,) can 
be determined (case (c)). 

As in the previous case, with the elementary transformation of the first 
kind, xX’ proves to be a support program, To verify this we must again 
consider the two possibilities arising when computing 4,. 

Let §,=6. Then 


#,= (€,¢>0) 
for some r=q+1. In this case 
x, =0, I# Jos B=1, 2, ..., r—l, r+l, oo, 8; 
30142) Pia a=1, 2, ..., fl ttl, ws. 
We now show that the square matrix of (s—1)-th order 


Gi, i, eee Gileos inset eee Gils 
eeeeee8 @® eoec¢ @®¢ # © © @® # *® 
Ax: = Gipsy: . By eigns Qi detae ° Geils 


Bees gee Bee lems Ure rlrers  * Frtile 


a4, cee Qijpe, Blogs ce aif, 


is regular. Indeed, the determinant of the matrix Ay. is equal, up to the 
sign, to the cofactor of the element a, y of the matrix Ay. Therefore 
? 


[Ax |= te,,|Ax| #0, 


since e,,#0. Thus X’ is a support program with the basis Ay, which con- 
stitutes a matrix of order (s—1)obtained from A x when the t-th row and the 
r-th column are crossed out, 

We now admit the possibility: 6,=6%. Then 


AM a 
.= a9 (5) <0) 


for some ![#i,(a=1, 2, ..., s). Obviously 
x,=0 for j# Jy B=1, % oo 85 
aut = > fori=i, a=1, 2, ...,f—1,#+1, ..., s, dal. 


We now show that Ax obtained from A, when the ¢t-th row is replaced by 
(@;),, Qj) s++» @, )IS a regular matrix, 
a 


According to (1.19), the vector (e,,, @4, «++, @)7 is the t-th column of the 
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matrix Ax, The coefficient of the t-th row when the vector (@,,, a) «+++ 4, ) 
is expressed in terms of the rows of the matrix Axis, therefore, equal to 

ss 3 

6 as 2 217031 #9. 
Further, applying Chapter 4, Theorem 2.1, to the vector (@,,, @), +++, @,) 
and to the rows of Ax, we conclude that Ay is a regular matrix, Therefore, 
in this case, X is a support program with the basis Ax., which is a matrix 
of order s. 

Since program X is assumed to be nondegenerate, 6,>0. Hence, accord- 
ing to (1.24), 
L(X') =L (X)—A,8>L (X). 


1-5, In short, given a support program X of problem (1.1)-—(1.4), we 
either establish its optimality (case (a)), or prove unsolvability of the prob- 
lem (case (b)), orelseimprove program X’ increasing the linear form (1.1) 
(case (c)). Thetransformationfrom program X to program XX’ is an itera- 
tion (step) of the method. Since the increase of the linear form (1.1) in 
each iteration is monotonic and the number of support programs of problem 
(1.1)—(1.3) is finite, the method is called finite. 

We emphasize that until now we have been dealing with the case of non- 
degenerate problems. Only this assumption ensures that 6, be positive and, 
consequently, the monotonic increase of linear form (1.1) in each iteration. 

The parameter §@, required for improving the given support program X 
is computed as a minimum of some set of numbers. When elementary 
transformation of the first kind is applied, this set is defined by 


*8 (8 — >0 
‘ai aang ar sker Siar oh (1.27) 
— 70 (e fora=1, 2, ..., s; 5’ <0), 
k 
For elementary transformations of the second kind, the set is defined by 
* 18 : 
SBOE GD vos 51 ey>0), (1.28) 
ti - 
Sa liti fora=1, 2, ..., s; 6?<9). 
t 


It can be easily verified that in the nondegenerate case the sets (1.27) or 
(1.28) each have one least element (see Exercise 3). This element, as has 
been shown in 1-4, uniquely defines the basis of the new support program 
X’. In the degenerate case, the minimum may be obtained simultaneously 
on several elements of the corresponding set (1.27) or (1.28). To deter- 
mine the basis of the new support program, one of the several elements 
equal to §, must be chosen. If the element is chosen arbitrarily cycling may 
arise, i.e., periodic return to the same basis. (This is known from ex- 
perience.) However, since cycling is fairly rare, in applied problems one 
of the least numbers in sets (1.27) or (1.28) may, nevertheless, be chosen 
arbitrarily. We shall not dwell here on the rule which guarantees against 
cycling in the solution of any problem of type (1.1)—(1.4). It suffices to 
note that this rule can be formed with the aid of the e-procedure used re- 
peatedly in the previous chapters. 

1-6. Some remarks on the simplex algorithm as applied to problem 
(1.1)—(1.4) are now necessary. It follows from (1.19) that the numbers 
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Css, Cnty oe, €,, CONStitute the #-th column of the inverse of the matrix 
Ay= Wi aiplle ' 
which is the basis of the support program xX. Hence, 
lesalls = lla, gle. 


We shall show that applying the inverse matrix |je,,\|, of the basis A, we 
can easily determine all the parameters required in an iteration for im- 
proving the program X. 

1. The parameters i,, a=1, 2, ..., s required for testing the program X 
for optimality satisfy (1.5) or, in matrix form, 


Here Ay=(A,, ..., Agh Ay = lle; aiglls Cy=(C)5- .., ¢;,). Multiplying both sides of 
(1.29) on the right by 
Ax’ = |lépalls) 

we obtain 

Ay= Cy legals, 
or, equivalently, 

A= : : a=l, 2, sie ey ae 
c= Bi Ciglte (1.30) 
2. If elementary transformation of the first kind is used for program 

improvement, the numbers x, B=1, 2, ..., s constituting the solution of sys- 
tem (1.9) must be determined. In matrix form, system (1.9) is written as 

AX = AN, 


where X = (x,,, Xgpy oe0s Xen)” Ab” (dj gs Biay oes a,,)". Hence, 
XO = AZ'AY?, 
Expanding the last equality we have 


Spy = Venta PH 2 veer be (1.31) 
a=) 


3. When elementary transformation of the second kind is applied, the 
analog of the vector X" is the corresponding column (the ¢t-th column) of 
the matrix |légall,. 

Thus, in order to carry out a single iteration of the simplex method 
without solving systems of equations, it is sufficient to know the inverse of 
the basis of the program to be improved. 

1-7. We shall now derive recurrence formulas for the elements of the 
inverse matrices of two successive (i.e., obtained one from the other ina 
single iteration) bases. 

We recall a proposition used repeatedly in previous chapters (see Chap- 
ter 5, 1-3). Consider a regular square matrix |d,/|, of order eg. Let 
Hayle . Ifin Id, ll, the /,-th column is replaced by the vector D,=(d 


to! 


d,,, ...,d,,)? the inverse lldeyHl, of the new matrix is determined from 
A di,tdig 
| ag (1,32) 
= 
dof sj 
dj." ‘ 
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b= 125. Say OF J Sl 2h oe O 


Here jy. j=1,2, ..., @ are the coefficients of the expansion of the vector D, 
in terms of the columns D,=(d,,, d,,, ...,d,,)7 of the matrix |{d,)||,: 


= > dB jdy. (1.33) 
Now substitute the vector D®=(d,,, d,,, d,,) in place of the /-th row in 
the matrix II ll, The corresponding inverse is again denoted by dell, - 


Since Nd, Ale is the inverse of the matrix lid slo we apply (1. 32) and (1.33) and 
obtain recurrence formulas for the elements of |{d; jl, and Id, Mp: 


~ dit, 5 . 
qiy— 7, doy, JF, 


~s (1.34) 
dij = dit, J i 
dein . 
i=], 2, eve, Q, jJ=1, 2, eve, Q, 
where 
4,.= > G,jdy,i=1,2,...50. (1.35) 


We now deal with the case when the new matrix is obtained from |jd,/|, by 
adding one row and one column to the latter. Schematically, the new ma- 
trix |ldi|,,, Can be represented as follows: 
aul 


ee | 


0) 
+-o- 


Niiloes =| 


(d+0). (1.36) ° 


Using this notation, we introduce the matrices 


d,,| 0 
1a. = | — 


>| 


Multiplication of the corresponding matrices gives, immediately, 


ae 
1 
(1) 


which is the inverse of IIa Jig The matrix ll; loss differs from Id; loss 
only in the last row, Applying (1.34), we obtain 


0 


1/d 


(Id; lees = 


e+1 = 


- d 
ld Mes | a 


d,, 
—d, ld 


4 2 
IId;jlors = 


for the inverse of lid, 18, Here the d,, are determined from (1.35). 
The matrix IZ:jl|,,, differs from 4 Noes only in the last column. There- 
fore, the elements of the matrix 


W4silleas — W4eslloes 


can be determined from the recurrence formulas (1.32) when applied to 
matrix |/d,\I¢},, which is known. 
We now compute the coefficients dj? in the expansion of the vector |Id; Ald? 


(4,,, 4,,,---,4,,, @)’ in terms of columns of Id slg: applying the explicit 


2154 
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expressions for the elements of the matrix 


Fit = de f=1, 2, ++ @ 
d\) = ; Q (1.37) 
1—5 Di 4,05, f=erl. 
y=1 
From formulas (1.32) and equalities (1.37), we obtain 
~ yd; 
( d+ for i, j=1, 2, wee, Q; 
d 
—> for i=g+1, j=1, 2, ..., @ 
-~ @ 
a;,= | a | (1.38) 
| —z for i=1, 2, ..., 0, J=et]; 
C 
= for i=j=e+1. 
( ae 


The parameters d,, and d;, are determined from (1.35) and (1.33) respec- 


tively, and 
d, =d > d, dy, d > dd 5. (1.39) 
y=1 yr 


Finally, consider the last case when one row (the i,-th)and one column 
(the j-th) arecrossed outfrom |ld;,||,. We thus obtain the matrix WVaeill,_, of order 
e—1. Letthe inverse of the new matrix IIdeslly_y be denoted by {dill . We in- 


troduce two auxiliary matrices of order @: ||d,,|ij”, which is obtained from 


lId, |, When the i,-th row is replaced by the j,-thunit row; and |ldj;[l¢? whichis ob- 
tained from |jd,,||? when the j-th column is replaced by the i,-th unit vector. 

The relationship between the inverses of the matrices [ld; j],, Ild,,1~? and 
Id; Mlg’, ll; ll? can be established proceeding from (1.34) and (1.32), respec- 
tively. 

If from the inverse of the matrix lid; Ig? we cross out the /-th column 
and the ,-th row, we obtain the desired matrix |[d;)\|"'. 

The above considerations lead to the following recurrence formulas for 
the elements of the matrices |}d; |, and |I4; /ll,_,: 


f di}, i<j), I<, 
d; i> 
diy =) ee Sele (1.40) 
d,, {+s l<j,, J=i,, 
dissin >), fei, 


where 

ras 7 d © d; - 

dy = 4; ,—F ix j,, j#i,. (1 41). 
Proof of formulas (1.40) and (1.41) is left to the reader (see Exercise 4). 

1-8. In the process of improving a program there are four ways of 

modifying the basis. Th-se are determined by the kind of the elementary 
transformation chosen and by whether the parameter 96, is equal to % or to 
6. The preceding formulas enable us to derive recurrence formulas re- 
lating the elements of the corresponding inverse matrices for each of these 


cases. 
1. Elementary transformation of the first kind generated by the 
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hyperplane x,=0 is applied, and 


The basis Ay of the new support program X’ is obtained from the basis 
Ayof program X when the r-th column is replaced by (4;,4, @j,4) «+5 Gin)? 

Let the inverses of Ay and Ax be denoted by Me; jll s and lez jll,, respectively. 

Applying (1.32), (1 33), we obtain 


€, j*ik ‘ 
j ei; Xrk ry i-r, 
eu = e,; (1.42) 
rj . 
are i=f, 
XpR 
be Ja Ne 2h 5G Se 


Here x,,— the coefficients in the expansion of the vector Af in terms of 
the columns of basis A,— are computed from formula (1.31). 

2. The same elementary transformation as in the previous case iS ap- 
plied, but now 


: Aw 
6, =o a ah 
4, 
In this case the basis Ay of program X’ is obtained by adding a row (a;,, 
Qt), -++sQij,, Gi) aNd a COluMN(Gik, Biky +++) Fiser Ge)? to basis Ax. Observe that 
the coefficients in the expansion of (ay, a4,, ..., @,) in terms of the rows of 
basis Ax are determined from 
s 
8a = DVaipgepa, a=1,2, ...,8. (1.43) 
B=1 


Applying (1.38), we obtain the following expression for the elements of 
ley [ise the inverse of matrix Ax: 


| ey tat O07) Jl 2, isc8558, 

et =| me for i=s+1, j=1, 2, ..., 8, (1.44) 
| a for i=1, 2, ..., s, j=s+l, 
[ = for i=j=s+l. 


Here 


8 s 
e, = An — > 60 Fi, Fk Die thy Xap: 
v=! y=1 
3. Elementary transformation of the second kind generated by the hyper- 
plane 


is applied, and 


Under these assumptions, the basis Ay of program X’ is obtained from Ax 
when the #-th row and the r-th column are crossed out. Therefore, to ob- 
tain the necessary recurrence formulas between the elements of |fe,,||, and 
lle; ls+, the inverse of the basis Ax, we may apply formulas (1.40) and 
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(1.41), Thus, 


é ° 
ey — if i<r, Jj<t; 
e e 
Cae Cis, J if i>r,j<t 
“a er, jie 1.45 
et, ju — if i<r, j>t; (1.45) 
e @ 
C41. Ina ry oan if i>r, J= t: 
(é, j=1, 2, ..., s—1). 


4. The last possibility arises when the elementary transformation of 
case (c) is applied, but 


The basis Ax of program X’ is obtained from Ay, by substituting the row 
(21;,, tj, --+, @j,) for the #-th row. Applying formulas (1.34), (1.35), and 
remembering (1.43), we obtain the following expression for the elements 


of the matrix jje’,||,, the inverse of Ax:: 


eyj— ath 0), JAE, 


Bo 
e,= sy j=t, (1.46) 
t 
i, J=1, 2, coos 8, 


The recurrence formulas (1.42), (1.44)—(1.46), and also the preceding 
relationships (1.30), (1.31) serve as the basis for the simplex algorithm. 
The reader is advised to compile this algorithm (see Exercise 5). It suf- 
fices to note that the principal part of the algorithm tableaus is filled with 
the elements of the matrix |je,;||, and has, thus, the order sxs, where 


max(p, g)<=s<min(m, n)=@q,. 


1-9. To conclude, we make the following remarks. To reduce problem 
(1.1)—(1.4) to canonical form, we must introduce m—p new nonnegative 
variables (equal to the number of inequalities in system (1.3)) and eliminate 
g old variables not restrained by the requirement of nonnegativity. The 
resulting problem has na+m—(p+q) nonnegative variables with m—gq equality 
restraints. Therefore, to solve the reduced problem by the second simplex 
algorithm, we have to deal with inverse matrices of order 9,=m—q. We 
mentioned at the end of 1-8 that the order of the inverse matrices used in 
solving the problem by the procedure described in this section is at most 


y,=min(m, n). 


When these methods are applied to universal digital computers, the in- 
verse matrix to be transformed in each iteration is generally stored in the 
working memory. The dimensions of this matrix are, therefore, a prin- 
cipal parameter necessary to decide whether the problem "fits" into the 
machine. In manual computations it is also advisable to keep the order of 
the matrix as low as possible. 

Hence, whether to reduce problem (1,1)—(1.4) to canonical form or to 
solve it by the method described in this section should be decided on the 
basis of a comparison between two numbers 


P, = min (m, n) 
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and 
Q, =m™—q. 


If p,<g,, reduction of the problem to canonical form is justified. If, how- 


ever, 
P, =P, = 2, 


it is expedient to apply the modified simplex method. 
Observe that for p=1 and g=1 the dimension of problem (1.1)—(1.4) can 
be reduced by eliminating 
t=min(p, q) 


variables (of the first gq variables) and omitting t equations from (1.2). 
With these transformations we obtain a new problem of type (1.1)—(1.4), 
where at least one of the parameters (p, q)iS a priori equal to zero. 

However, when reducing the number of variables the restraint matrix 
Ais changed. Therefore, it may occur that the restraint matrix A’ of the 
equivalent problem will be less convenient for analysis than the matrix 4; 
for instance, the matrix A may have far more zero elements than the 
matrix A’. In such cases the transformation for reduction will, obviously, 
complicate computations; slight reduction in the dimension of the problem 
will, to a large extent, increase the laboriousness of a single iteration, If 
the matrix A has no useful features and p, g=1, it is advisable to eliminate 
part of the variables in problem (1.1)—(1.4). 

These remarks emphasize further the preliminary analysis of problem 
restraints. 

1-10. There are some classes of linear-programming problems which 
are best solved in their original form. We shall give some of them. 

Consider the general linear-programming problem in a nonnegative 
variables with m equality restraints (m<n), The dual problem contains 
variables not restrained by the nonnegativity requirement, and has a in- 
equality restraints. Therefore, for the dual problem 


Q,=min(m, 2)=m;, 9, =A—m., 


If m<n—m it is advisable to solve the dual problem without modification, 
In Chapter 6, § 8 we showed that the dual simplex method is an application 
of the simplex method to the dual problem. It is obvious that the dual prob- 
lem is then solved according to the procedure described in this section. 
Since none of the dual-problem variables is restrained by the requirement 
of nonnegativity, the solution is based only on elementary transformations 
of the second kind with 6,=6 throughout. 

Consequently, only case(d) is realized when improving the program of 
the dual problem: the bases of the successive support programs differ only 
in one row (or, in terms of the primal problem, in one restraint vector). 

As another example, consider the statement of the general linear-pro- 
gramming problem proposed by L. V. Kantorovich /64/ (in application of 
the industrial-planning problem). 

An industrial plant, or a group of plants must release m, different out- 
put products in a predetermined proportion (in a given assortment) speci- 
fied by the numbers i, A,, ..., Ag, In the process m, production factors are 
used, whose resources comprise 8, },, .-., m,units, respectviely. By as- 
sumption, 6, >0(/=1,2,...,m,). If 6,=0, we take it that the i-th factor isa 
semifinished product of the given plant. 
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There is a certain number (a) of different production modes. The j-th 
production mode is characterized by the vector 


A, = (a; Gj aitins On), 


where m=m,+m,. The first m, components of the vector A, determine the 
consumption of the corresponding production factors by the /-th production 
modes in unit time (unit intensity). The other m, components give the quan- 
tities of the corresponding output products produced in unit time by the 
given production mode, The principal purpose of the plant is to arrive at 
maximum possible output observing the proportions specified and drawing 
upon the resources available. 

A production program is a set X=(x,, x,, ..., x,) of utilization intensities 
of the various production modes, 

The problem of determining the optimal production program can be 
stated mathematically as follows. 

Determine the vector X=(x,, x,, ..., *,, V)Ssatisfying the restraints 


n 


Pu dmst 1Xje AV, i=l, 2; ees ms (1.47) 
=1 
n 
p2 AijxX, ad, i=1, 2, ..., mJ (1.48) 
}=1 

x,=0, Jal, 2) veg Rj (1.49) 


and having a maximum possible (n+1)-th component 
L(X)=V. (1.50) 


Here V is the number of assemblies of the output products which can be 
released under the program X=(x,, x,, ..., *,), where by assembly we mean 
a set of these products in the respective amounts 4,, Ay, ...; Ag. 

Problem (1.47)—(1.50) has n+1 variables, n of which are assumed non- 
negative, and m=m,+~m, inequality restraints. In this case 


Q,=min(a+1, m,+m,), 
P, =m, +m,—1. 


In general n<m,+m, and, consequently, 9,<9,. This indicates that it 
is advisable to solve problem (1.47)—(1.50) in its original form, i.e., as 
originally stated. 

In the next section, where we consider a modification of the simplex 
method based on simultaneous introduction of several restraint vectors into 
the basis, we shall again encounter the necessity of applying the technique 
discussed here, 

Finally, consider the class of problems of best uniform approximation 
of functions defined on a finite system of points by linear combinations of 
a fixed set of functions. Application of the simplex method as described in 
this section to problems of this class leads to the method suggested by 
S.I. Zukhovitskii /60/. 

In this section we dealt only with the simplex method. This, however, 
does not mean that other general methods of linear programming are only 
applicable to problems in canonical form, Each general linear-program- 
ming method can also be adapted to problems of type (1.1)—(1.4) in a manner 
similar to that shown in this section for the simplex method. 
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§ 2. Modification of finite methods 


2-1. Consider the general linear-programming problem in canonical 
form. 

The simplex method entails a successive examination of adjoining sup- 
port programs of the problem whose bases differ only in one vector. The 
vector to be introduced into the basis is chosen from among the restraint 
vectors with negative relative evaluations. There are usually several 
restraint vectors with this property, and each is suitable for inclusion in 
the basis. However, following the recommendations of the simplex method, 
we choose only one such vector. The question is whether the simplex 
method can be modified so that several suitable restraint vectors can be 
introduced simultaneously into the basis. The same question can be posed 
regarding the dual simplex method. 

When the dual simplex method is used, the solution procedure entails 
examination of adjoining pseudoprograms,. The bases of two adjoining 
pscudoprograms differ only in one restraint vector. To obtain a new pseu- 
doprogram, we compute the coefficients in the expansion of the constraint 
vector in terms of the basis vectors of the given pseudoprogram. The 
basis vectors corresponding to negative coefficients can be eliminated from 
the basis. Although in most cases there are several such basis vectors, 
only one is eliminated. Another question is whether it is possible to eli- 
minate from the pseudoprogram basis, several suitable vectors simul- 
taneously. 

The present section deals with these questions. The answer to both 
questions is positive, and for problems with large dimensions application of 
the modified technique outlined in what follows has concrete advantages. 
The principal part of the section is devoted to the simplex method; modifi- 
cation of the dual simplex method is given more briefly. Some of the 
results of this section can also be found in /2/. 

2-2. The entire discussion is with reference to the linear-programming 
problem of maximization of the linear form 


n 
subject to the conditions — 
D4, ,x,= 5, i=1, 2, eee m; (2.2) 
jz 
x,=0, JS1, 2) Sat N. (2.3) 
Let X, = (x, x, ..., x)be a support program of problem (2,1)—(2.3) with 
basis A,, A,, ...,As, Here, as always, A;=(a,, a, ..., @,,)7 is the j-th 


restraint vector of problem (2.1)—(2.3). 

We rewrite system (2.2) in the equivalent form expressing the basis 
variables of the given program in terms of the other unknowns of the sys- 
tem (see Chapter 5, 4-1): 


a a up b=1, 2, ..., m (2.4) 


Here %,), Xs, »0+, Xq, are the coefficients in the expansion of the vector A, in 
terms of the basis vectors A,,, Ay, s001 Ag of program X, (/=0, 1, ..., @ A, = B). 
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Let /x, be the set of indices s,, s,, ..., s, corresponding to the basis vectors 
of program X,. 


If we introduce the expressions for the variables x,,i=1, 2, ..., méinto 
(2.1), we obtain (see Chapter 5, (4.7)) 
L =L 4 _ A r 2.5 
= LX) — 2p My (2.5) 


where 


| 
4,= Bryta—ep jJ=1, 2, ooo, A, 


If A,=0 for /€/x,, program X, is optimal. Now let some of the A, be 
negative. Following the simplex method, we choose one of these parame- 
ters, say A,<0. The value of the extrabasis variables x, begins to in- 
crease and the other extrabasis variables retain their zero values. The 
value of the basis variables is determined from formula (2.4). 

Let the current value of the &-th variable be 6, then restraints (2.3) are 
satisfied if and only if 


X,,—0x,,20, $=, 2, ...,.m; 00. (2.6) 
When conditions (2.6) are satisfied, the vector 
X (0) = (x, 0), x, (6), «--. ¥, (8), 


where 
Xjg—OXy, Js; ($=1, 2, ..., m), 


6, jJ=k, 
xis) in other cases, 


is a feasible program of problem (2.1)—(2.3). 
The transformation from program X, to program xX(8) increases the 
linear form (2.1) by 
L(X())—L (X,) =— A,B. (2.7) 


It is obviously best to choose such a value 9, of the variable § so that the 
linear function (2.7) is maximized subject to (2.6). The value 6, is thus a 
solution of problem (2.6), (2.7) in one variable (6), 

Now let there be several negative A, We shall try to introduce several 
of the restraint vectors A, (/¢/x,, 4,<(0) simultaneously into the basis of 
program 4X%,. 

Let E be the set of indices of the vectors A, chosen to be introduced into 
the basis. Let 0,(u€£) be the current value of the variable corresponding 
to the vector A,. It follows from (2.4) that for fixed #,, hE 2, the values of 
the basis variables X,of x, are equal to 


Xig > x,,6,. 
pee 
The parameters 0,, hE Z, thus define a transformation of program X, into 


vector X (8) =(x,(0), ..., x, (0)), where 


BEE 
6, JE€E, (2.8) 
0 


Xie D2 x;,0,, J=s,((=1, 2, eos m), 
in other cases, 


Here @ indicates the set of parameters §, for p€BE. 
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By definition, the vector X(6)satisfies equalities (2.2) for any 0,, hE Ex. 
Therefore, the inequalities 


Xie— 4,0, t=1, 2, ..., m, (2.9) 


pee 


and 6.20, pee, (2.10) 


are necessary and sufficient for the vector X(6) to be a feasible program of 
problem (2.1)—(2.3). 
It follows from equality (2.5) that 


L(X (6)) —L (X,)=— 2, A,6,. (2.11) 


The parameters 0,, #€2, should be so chosen that under transformation to 
the new program X(6), linear form (2.1) acquires the maximum possible in- 
crement, Hence, the required values of the parameters 0,, pEf, are deter- 
mined by maximizing (2.11) whose variables are subject to conditions (2.9) 
and (2.10). 

We shall refer to problem (2.9)—(2.11) as the auxiliary problem 
associated with program xX, and set £. Let # be the number of variables in 
the auxiliary problem, equal to the number of elements in set B The mxt 
restraint matrix Ay, ,of the auxiliary problem is constructed of elements «,,. 

2-3. Problem (2.9)—(2.11) is not given in canonical form. It can, there- 
fore, be solved by the modified simplex method given in the preceding sec- 
tion. According to the general definition in 1-1, to each support program 
of problem (2.9)-—(2.11) there corresponds a regular square submatrix of 
matrix Ay», the basis of the support program. The order t of the basis 
ranges from 0 to min(#, m). The number of variables ¢ in the auxiliary prob- 
lem (2.9)—(2.11) is usually taken less than m, Therefore 


O<t<t, 


Observe that the case t=0 (the set of basis elements is empty) is possible 
in this problem. It corresponds to the support program 6,=0 for pe€B&. 

Taking §,=0 for p€& as the initial support program, we solve problem 
(2.9)—(2.11) by the simplex method. Since A,<0 for pé€E&, the first itera- 
tion cannot terminate in case (a). Hence, solution of the auxiliary problem 
is terminated either when unsolvability of the problem is established, or 
when an optimal support program is obtained with basis oforder t, where 


1xrt<t., 


The first possibility implies unsolvability of the initial prope (2.1)— 
(2.3). Consider the second possibility. 

Let the basis of the support solution 6. pee, of the auxiliary problem 
comprise the elements which in matrix A,,, are the intersections of the 
rows and the columns with the indices,r,, r,, ..., r,andk, k,, ..., &, res- 
pectively. Let X’=(x;, x, ..., %,) be a program of problem (2,1)—(2.3) deter- 
mined from formula (2.8), where the parameters 6, are replaced by 6, (» € E). 

Obviously 


é 
%= Xie 2 £148, =0 for i=r,, Pes eee, r. 


xj=0, if I¢ly, and f#h, by ..., hy. 
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Let /x- be the set of indices of restraint vectors obtained from /y, when 
the indices s, fori=r,,r,,..., 7, are replaced by k,, &,, ..., ,. 

To show that X’ is a support program, we must prove linear independence 
of the vectors A, j€/x’. 


Let -1 T 
X= Ax,A,= (x,y, Xp eee, Xm) ’ 


where Ax,is a matrix constructed of the basis vectors of program X,. Take 
the determinant Dx, constructed of the components of X,, j€/x. Expanding 
the determinant Dy. in columns 


X;, = (0, 0, ere 0, 1, 0, eee 0)7, 
| i | 
l<si<cm, i#n, 1<aArt, 


we have 
Dx: — + D,, 


where D, is the determinant of the basis of the solution of auxiliary prob- 
lem (2.9)-(2.11). However, D,#0. Hence Dx #0, and the vectors X,, j€/x: 
are linearly independent. 

Further, since A,= Ay, X; and |Ay,| #0, we conclude that the system 4, 
for jé/x: is linearly independent. We have thus established that xX is a 
support program. 

The solution of the auxiliary problem gives 

(a) the indices /€E of the vectors A, which may be introduced into the 
basis (j=k, A=], 2, ..., T)s 

(b) the indices s, of the vectors A, to be eliminated from the basis 
(é=r,, A=1, 2, ..., 7). 

Observe that the number t of vectors introduced into the basis may be 
less than the number f¢ of elements in set E, However, we have seen that 
t=>1. Therefore, at least one of the basis positions will have a new vector. 
In most cases the number of basis positions into which new vectors are 
introduced is greater than one. All the operations needed for transforma- 
tion from program X, to program X’ will be called the large iteration. 
A single step of the simplex method applied for solving the auxiliary prob- 
lem will be called asmall iteration. Before the large iteration as- 
sociated with the support program X’ can be started, we must either know 
the coefficients in the expansion of the restraint vectors in terms of the 
basis of program X’ (first algorithm) or the inverse of the basis matrix of 
program XX’ (second algorithm). 

2-4, Consider the second algorithm, We shall derive recurrence for- 
mulas for the elements of the matrices 


Ax, =lleisllas Ax! = (let lla 
The matrices Ay, and Ax, i.e., the bases of programs X, and X,. differ 
in + vectors. To simplify notations, let us assume that the first. posi- 
tions in Ay, are replaced, i.e., 7,=1, ..., r,=1. This can always be attained 


by performing a suitable permutation of the columns of matrix Ay, and the 
rows of Ax. Thus, 


Ax =(Akp,, Any eon A, Ay. ooo 9 A,,). 
Multiplying Ax on the left by Ax!, we have 
AX,Ax! =(Xgy Xue cons Xa Crt coos Mw) (2.12) 
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where 
X= AX Ap €j=(0, 0, ..., 0, 1,0, «66, 0)%, 
aac aaa 
The matrix (X,, X4, ..-, %) is constructed of the coefficients in the expan- 
sion of the vectors A,, Ag, «>; Ay, introduced into the basis in terms of the 
basis vectors of program X,,. 

Let B, be the square matrix of order + constructed of the first + rows 
of the matrix (X,, ..., %,) (the elements of B are the coefficients in the ex- 
pansion of the vectors introduced into the basis which correspond to the 
vectors eliminated from the basis). Let B,_,, be the (m—t)xt, matrix com- 


—T%, t 


prising the remaining m rows of the matrix (X,, X,, --., X,). 
With these notations equality (2.12) can be rewritten in the form 
B O.2 
Ax, Ax: = = rs ’ (2.13) 
mnt, F | ae 


where O is the tx(m—rt) zero matrix, and E is the unit matrix oforder 


Tt) Mt m= 
m—t.. By direct matrix multiplication, we can easily verify that 
B O.,a-:|| B;" 0., n- 
D= Ty Bot = os = ty Mt (2.14) 
ieee | fae — By, .Be Eo 
Applying (2.13) and (2.14), we obtain 
DAx,Ax' = E,. 
Hence 
Ax = DAx. (2.15) 


Equality (2.15) establishes the desired relationship between the matrices 
Ax and Ax! 

We divide the matrix Ax,=|e,,j, into two: (1) a matrix comprising the 
first t rows of Ax;; (2) a matrix comprising the remaining m—t rows. The 
order of these matrices are txm, |ej;, m and (m—t)xm, le:,,, jlm-x, » TeSpecti- 
vely. In the same way, we divide the matrix Ax'=|le;;|,, into the matrices 
flees le, m and jess, jIm-c, The special structure of matrix D, as defined by 
(2.14), makes it possible to rewrite formulas (2.15). Thus: 


betes, | eae ees, jlo n— Bmn=t, B;' fe; /t., m (2.16) 
becibe, m= Be hersh, we (2.17) 


These formulas are generalizations of recurrence formulas (1.21) in 
Chapter 5, which were widely used in stating the algorithms of linear-pro- 
gramming methods, The formulas there are a particular case of (2.16), 
(2.17) for r=1. 

We now give a brief outline of the sequence of operations involved in 
each large iteration under the second algorithm, 

To begin with we have the support program X, with the basis. (A,, A 

-» As) =Ay, and the inverse matrix 


$2) oer y 


Ax, =[eijhn 
We compute 


m™ 
A= > Coq Fats i=l, 2, @0es %, 
and then bes 


A,= 2 a, A;—Cp, té Ix,. 


448 


If all the A. are nonnegative, X, is an optimal program. If some of the A, 
are negative, we proceed with the construction of the new support program 
X". We choose several (t) restraint vectors which may be introduced into 
the basis (the set of indices of these vectors is denoted by £). The expe- 
diency of introducing a vector A, into the basis is generally determined by 
A, the evaluation of A, with respect to the previous basis, However, the 
system of vectors to be introduced into the basis can be chusen also on the 
basis of other considerations. The choice of the number ¢ of the elements 
in set E depends, to a considerable extent, on the computational means 
available, and generally t<1 m(we shall return to this problem in 2-5). 

The set £ specifies auxiliary problem (2.9)—(2.11) in # nonnegative 
variables with m inequality restraints. The parameters x,, of the auxiliary 
problem are computed with the aid of the matrix |e;,],,: 


+5, = 2 Boje in UE £). 


Problem (2.9)—(2.11) is solved by the modified simplex method described 
in the preceding section, In the initial program of the auxiliary problem, 
all the variables are zero, and the basis of this program is empty. In each 
iteration of the auxiliary problem we compute (from recurrence formulas) 
the inverse of the basis matrix of the corresponding program, The solu- 
tion procedure is terminated only when unsolvability of the auxiliary pro- 
gram is established, or its optimal program with basis B. is found. The 
former case implies unsolvability of problem (2.1)—(2.3), and in the latter 
we pass to a new support program of the problem in question, 

Let the basis B, of the optimal program 6,, »€£ of problem (2,9)—(2.11) 
comprise the elements of the restraint matrix of the auxiliary problem 
which are the intersections of the rows and the columns with the indices 
lily «7, and k&, k,, ...,& respectively. In this case the basis A,y-of the 
new support program X’ of problem (2.1)—(2.3) is obtained from Ay, when 
the vectors 4f,,/,, ...,7, are inserted in the positions A,, ..., Ag; the 
components of program X’ are computed from (2.8), where 6, are replaced 
by 6, 

Together with the optimal program of the auxiliary problem we also 
compute the matrix By’, Therefore, the inverse 


Ax! =[er; fe 


of the basis matrix of program X’ can be determined from recurrence for- 
mulas (2.16), (2.17). As a result we obtain program X’ and matrix Aj’, 
both of which are essential for proceeding with the next large iteration. 

2-5. We now make some comments which will help the reader to see 
more clearly the essence and the advantages of the modified simplex method 
described here. We consider first the geometrical interpretation of the 
large iteration, limiting the discussion tothe first geometrical interpreta- 
tion. The support program X, corresponds to a vertex of the polyhedral 
restraint set of problem (2.1)—(2.3) (the set M), Having established non- 
optimality of program X,, we form the set 2 of the indices of restraint 
vectors chosen to be introduced in the new basis. The set E constitutes 
the auxiliary problem of the given iteration, The vertex X is the intersec- 
tion of hyperplane (2.2) and 


x,=0 for J&/y,,. (2,18) 


Having isolated the set B, we replace restraints (2.18) for /EE by x, 20. 
Hence, the polyhedral restraint set Mz of the auxiliary problem coincides 
with the face of the set M passing through the vertex X,. The auxiliary 
problem entails maximization of the linear form of the principal problem 
on the set M,. The dimensionality eg of the set M, is at most ¢. In the non- 
degenerate case 9,=f. The point X,is a vertex of the set M,. The auxiliary 
problem is solved by examining the adjoining vertices of the set Mg, start- 
ing with the vertex xX, This procedure is carried out following the rules 
of the simplex method and therefore (in the nondegenerate case) the linear 
form of problem (2.1)—(2.3) increases monotonically. 

The transition from a given vertex of the set M, to another (adjoining) 
vertex is a small iteration. Each vertex of M, is also a vertex of the set 
M. Therefore, having derived an optimal support program of the auxiliary 
problem, we arrive at a new support program X’ of the principal problem, 
the result of the given large iteration. 

For t=1and a nondegenerate program X,, the set M,;is an edge of the 
set M. 

The bounded set M, has two vertices. The auxiliary problem is, there- 
fore, solved by a single iteration. This case applies when the unmodified 
form of the simplex method is employed. 

The auxiliary problem is obtained from the principal problem in which 
some of the variables are taken to be zero. The zero variables for each 
auxiliary problem can be chosen by different methods. 

When the simplex method is used, one is naturally guided by the evalua- 
tions A, of the restraint vectors A, with respect to the given basis taking as 
zero those x, for which A,>A, where A is some number. The Hungarian 
method and its modifications are also based on solving a sequence of auxi- 
liary problems. Here, however, x,=0 are chosen proceeding from the 
current program of the dual problem. 

The modified simplex method described here removes some of the re- 
strictions imposed on the dimensions of the problem by the capacity of the 
operative memory of the computer used for the solution. The restraint 
matrix with large dimensions can be stored inthe external memory of the 
machine. 

During the large iteration it is advisable to store in the working me- 
mory of the computer only the restraints of the auxiliary problem whose 
dimensions are mxt. This makes it possible to carry out all operations re- 
lated to the external memory at the end of the large iteration. A lot of the 
time required by the machine for solving a problem is spent in addressing 
external memory devices. Therefore, reducing the number of times ne- 
cessary to address these devices greatly reduces the time needed for solution, 

When forming the auxiliary problem, we may vary the parameter ?, the 
number of variables of the auxiliary problem, The dimensions of the 
operative memory bounds ¢ from above. In the first stages of solving the 
principal problem, when the program available is still far from the optimum 
the increase in the linear form in a single iteration will be larger, the 
larger ¢. Therefore, increasing ¢, we reduce the number of large itera- 
tions and therefore the number of addresses to the external memory. 

This, however, obviously increases the number of small iterations re- 
quired for solving the auxiliary problem. Increase in the dimensions of the 
auxiliary problem affects only slightly the time required for solving the 


? 
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entire problem, since small iterations do not entail addressing the external 
memory. Thus, in the first stages of the solution ¢ should be taken as 
large as possible. In the last stages of solving the principal problem, it is 
advisable to take ¢ smaller, since then the increase in the linear-form does 
not depend much on..t, 

When problems with large dimensions are solved by finite linear-pro- 
gramming method, the number of iterations may be considerable. There- 
fore, to obtain a solution with some degree of accuracy we should 

(a) either carry out all computations to a sufficient number of significant 
figures, 

(b) or, at fixed intervals, compute the current parameters directly, 
without resorting to the recurrence formulas. 

The applicability of (a) is limited by the dimensions of the operative me- 
mory. If (b) is carried out, the time for solution is substantially increased. 
The modified simplex method to a large extent reduces the effect of round- 
ing-off errors, This is so because in each large iteration we are concerned 
with a problem with relatively small dimensions. When passing to the suc- 
cessive large iteration, it is sometimes expedient to obtain the problem 
parameters by computing directly the corresponding inverse matrix. 

2-6. We now consider the possibility of simultaneous elimination of 
several vectors from the pseudoprogram basis in the dual simplex method. 
We give but a brief description of the modified dual simplex method. Never- 
theless the reader, having thoroughly acquainted himself with the preceding 
articles of this section, will be able to fill in the missing details. 

Let the pseudoprogram X,=(x, ..., x) of problem (2.1)—(2.3) be given, 
its basis beingA,, A,, ...,As,. The pseudoprogram xX corresponds to some 
support program Y,=(y,”, y, ..., y®) of the dual problem of problem (2.1)—- 
(2.3). 

Let A= DxijAy  1=0, 1, cee 


m 


m 
A,= a; ()_o>6 — C3,X%; 7 Cy, =. > ree fn. 


Thus xO =x,, i=1,2,..., m xf =0 for /#s;. 


Let some of the basis components x;,of the pseudoprogram X, be negative. 
This indicates that X, must be modified by refining the system of preli- 
minary evaluations of Y,. 

We take some pseudoprogram-basis vectors with negative basis compo- 
nents, Let these vectors occupy the positions r,,r,, ..., 7, (#<m) in the basis. 
To establish which of these vectors should be eliminated from the basis and 
which vectors should be introduced, we must solve the following auxiliary 
problem: 

Maximize the linear form 


n 
— 2 Ars (2.19) 
subject to the conditions 
3 ty He B=, Tay cee Mp (2.20) 
=1 
x,=0, J=1, 2, @oey a. (2.21) 
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Let the components of the vector X, = (x, x, ..., xn’)be defined as 


xO = xy) if J=S fori=r,, Fer coon Vey 
J 0 in other cases, 


Obviously, X, is a pseudoprogram of problem (2.19)—(2.21) with the basis 
Ay: N=F,, fay «++. % Therefore, to solve problem (2.19)—(2.20) it is ad- 
visable to apply the dual simplex method starting with the pseudoprogram X,. 

Having solved problem (2.19)— (2.21), we obtain its optimal program with 
the basis A,, Ag, «++, Arp 

The basis of the new pseudoprogram of problem (2.1)—(2.3) is obtained 
from the previous basis by inserting the vectors A,, A,, ..., Ae, in the posi- 
tions ¢,,7,, -..,7;. Observe that some of the vectors A, may coincide with 
the vectors A,, \=r,, ..., 7; the corresponding positions of the basis re- 
main unchanged. This modified dual simplex method may be applied both 
under the first and under the second algorithms. Furthermore, it has the 
Same computational advantages as the corresponding modification of the 
simplex method (see 2-5). 


§ 3. Classification of the finite methods of 
linear programming 


3-1. In previous chapters we studied in detail three finite linear-pro- 
gramming methods: (a) the simplex method, (b) the dual simplex method, 
(c) the Hungarian method. 

In different chapters we mentioned other finite methods and possible 
modifications of the three methods described in this book. It seems ad- 
visable to consider the various finite methods from one viewpoint, compar- 
ing the basic premises of each, the structure of each iteration, and the 
sequence of transitions between successive iterations. 

The classification of the finite linear-programming methods admits of 
various approaches. ‘The methods can be divided into groups depending on 
whether the primal or the dual problem, or both problems of the dual pair 
are solved. Linear-programming problems have obvious geometrical and 
economic interpretations, The methods available for solving linear ex- 
tremum problems may also be classified according to each of these inter- 
pretations, There are also other formal principles according to which the 
finite methods can be analyzed from a single viewpoint. 

The necessity of a unified approach to the various linear-programming 
problems led us to repeat, in the following, some general considerations 
which have been discussed in various chapters with reference to each 
method separately. 

3-2. Finite methods of linear programming can, obviously, be distin- 
guished according to whether the solution of the problem is obtained by 
examining programs of the primal problem, the dual problem, or both prob- 
lems of the dual pair simultaneously. 

Using this distinction as the basis of classification, the simplex method 
and its various modifications should form the first group, in which the op- 
timal program is obtained by examining support programs of the primal 
problem, The solution procedure starts with an analysis of the initial 
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Support program, In each iteration we transfer from one support program 
to another which, generally speaking, increases the value of the linear form. 

The dual simplex method and its various modifications, obviously, form 
the second group of finite linear-programming methods in which the prob- 
lem is solved by examining programs of the dual problem. The solution 
procedure starts with an analysis of the given support program of the dual 
problem. In each iteration we pass from a given support program of the 
dual problem to another or, equivalently, from a given pseudoprogram of 
the primal problem to another. The transition to a new program of the dual 
problem will, generally speaking, decrease its linear form. In other 
words, the transition to a successive pseudoprogram will decrease the 
linear form of the primal problem. The computational procedure of the 
dual simplex method is stated in terms of the primal problem. 

The Hungarian method and its modifications, in our classification, form 
the third group of methods, in which both problems of the dual pair are 
dealt with. The solution procedure starts with an analysis of the given 
feasible program (not necessarily a support program) of the dual problem, 
In each iteration we pass from a given program of the dual problem to 
another. Simultaneously, in the augmented problem, which differs from the 
primal problem in its linear form and in having additional variables with 
unit restraint vectors, we pass from a given support program to another. 
In terms of the primal problem, each iteration of the Hungarian method is 
a transition from a given quasiprogram to a next quasiprogram with a 
smaller residue, 

The characteristics of methods of the third group stand out most clearly 
in one of the modifications of the Hungarian method, namely the method of 
bilateral evaluations. Here the solution begins with a feasible (not ne- 
cessarily a support) program of the dual problem and a support program of 
the primal problem. In each iteration we pass to a new feasible program of 
the dual problem and a new support program of the primal problem, In 
the method of bilateral evaluations each iteration reduces the program 
residue, i.e., the difference between the values of the linear forms of 
the dual and the primal problems on the corresponding programs. 

These considerations show that the three methods described in this book 
can be taken as principal representatives of the three groups of finite me- 
thods reflecting essentially different approaches to the solution of linear- 
programming problems. 

3-3. In Chapters 1, 2, and 3 we discussed two geometrical interpreta- 
tions of the linear-programming problem and its dual problem. ‘The second 
interpretation is more convenient for reviewing and comparing the various 
finite methods. When applying this interpretation we assume the problem 
to be given in canonical form. 

In the (m+1)-dimensional space of points U=(u,, ..., #,,,) we consider a 
convex polyhedral cone K andaline Q,. The cone K is spanned by the 
augmented restraint vectors A,J=1, 2, ...,”). The line Q parallel to the 
Oum+i1-axis passes through the point B=(b,, ..., 5,, 0)defined by the constraint 
vector. A correspondence is established between the (m+ 1)-dimensional 
space of the points U and the n-dimensional space of points X=(x,, ..., X,), 
according to which the cone K corresponds to points Xx>0, and the line Q 
is the image of points with zero residue vector 


E=B—AX 
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(here A is the restraint matrix and 8 the constraint vector). Thus, the 
intersection of the line Q and the cone K corresponds to points X satisfying 


the conditions 
X>0, AX=B, 


and is, consequently, the image of the domain of definition of the linear 
form. 

In a solvable linear-programming problem, the intersection of the line 
Q and the cone K may either be a segment ora ray. For the sake of sim- 
plicity we shall assume that the intersection is a segment. The coordinate 
Un,, of the point U specifies the value of the linear form on the corres- 
ponding vector X. The highest point of intersection M of the line Q and the 
cone K corresponds to the maximum of the Jinear form, and the lowest 
point m, to the minimum of the linear form in its domain of definition, Thus, 
geometrically, the linear-programming problem amounts to finding the 
highest (in maximization problems) point of intersection of the line Q and 
the cone K. 

The hyperplanes spanned by the m linearly independent augmented re- 
straint vectors and which intersect the segment Mmcorrespond to support 
programs of the problems. The hyperplanes spanned by the m linearly in- 
dependent augmented restraint vectors which do not intersect the segment 
and extend above the cone K correspond to support programs of the dual 
problem. 

It can be proved (see Exercises 8 and 9) that if the segment Mm is the 
image of the domain of definition of the linear form of the primal problem, 
the domain of definition of the linear form of the dual problem is mapped 
onto the ray MQ, the half of the line Q extending upward from the point M. 

The segment Mm contains a finite number of points which are images of 
the support programs of the primal problem, Similarly, the ray MQhas a 
finite number of points corresponding to the support programs of the dual 
problem. 

To surface points of the cone K, with the exception of M and m, corres- 
pond the vectors X with nonzero residue vectors, The surface of the cone 
can be divided into an upper and a lower part. The point U on the cone 
surface belongs to the upper part of the surface if a hyperplane separating 
the cone K and the positive semiaxis Ou,,,can be drawn through the ray 
OU. The lower part of the surface is defined analogously. 

Constructions similar to those described in Chapter 7, 3-1, make it 
possible to isolate a finite number of points on the upper part of the conical 
surface corresponding to quasiprograms of the maximization problem, 
whereas points of the lower part of the surface are images of quasiprograms 
of the minimization problem, 

Proceeding from these geometrical considerations we can summarize 
the specific features of the various finite linear-programming methods. 

The simplex method and its modifications correspond to examination of 
programs of the primal problem. In each iteration we obtain a new support 
program which, generally speaking, ensures a higher (in maximization 
problems) value of the linear form. Hence, the simplex method can be de- 
scribed by the moving from point to point along the segment Mm (images of 
the support programs of the primal problem) upwards to the point M. 

The dual simplex method and its modifications correspond to examina- 
tion of the support programs of the dual problem. Each iteration specifies 
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a transformation from a given support program of the dual problem to 
another support program which, generally speaking, decreases the linear 
form, The dual simplex method is described by moving from point to point 
along the ray MQ (images of the support programs of the dual problem) 
downward to point M. 

Each iteration of the Hungarian method transforms a given quasiprogram 
into another quasiprogram with a smaller residue. Hence, the Hungarian 
method, when applied to the maximization problem, corresponds to moving 
from point to point along the upper part of the conical surface (images of 
the quasiprograms) which eventually lead us to the optimum. 

There are other finite linear-programming methods which produce the 
maximum after a more complicated procedure. In particular, from the 
viewpoint of this geometrical classification of the methods, the method of 
bilateral evaluations discussed in Chapter 7, § 6 should be classified not 
as a modification of the Hungarian method, but rather as a combination of 
the simplex and the dual simplex methods. In the method of bilateral eva- 
luations the optimum is approached along the line Q from without and from 
within the cone. 

The simplex method thus corresponds to motion toward the optimum 
from within the cone, the dual simplex method, to motion from without the 
cone, and the Hungarian method, to motion over the surface of the cone. 

In all three cases the optimum is approached through points belonging to a 
finite set of points. 

The geometrical classification thus suggestes that the three methods dis- 
cussed in the book can justifiably be considered as the principal finite 
methods of linear programming. 

3-4. The economic interpretation of the linear-programming problem 
given in Chapter 1, 7-2, and of the dual problem given in Chapter 3, 1-5, 
outline a somewhat different approach to the classification of finite methods. 

We have seen that when a problem is stated in economic terms, it is 
better to write the problem restraints in the form of inequalities. The 
primal problem with the linear form 


bs 3.1 
& CX, (3.1) 
and the restraints 
Bape) < dy i=1, 2, were Mm, (3.2) 
=1 
x, 20, J=1, 2, oes? Nn, (3.3) 


is interpreted as a scheduling problem for a plant (scheduling of times or 
utilization intensities of various production modes from several methods 
given in advance). The desired program must ensure maximum output of 
some homogeneous commodity with given resources of various production 
factors. 

The following notations were adopted in (3.1)—(3.3): 

x,, the time during which the plant operates under the /-th production 
mode (the utilization intensity of the j-th production mode); 

a,,, the number of units of the i-th production factor consumec under the 
j-th production mode in unit time (with unit intensity); 

6,, the resource of the i-th production factor; 
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c,, the number of units of the output commodity released in unit time 
under the /-th production mode. 

In some cases it is more convenient to interpret c, as the valuation 
(price, cost) of the output commodity released in unit time under the /-th 
production mode. Depending on the definition of the coefficients c,, the 
linear form indicates the total volume or the total valuation (price, cost) of 
the output commodity. 

Each production mode is characterized by the augmented restraint vector 
A, The first m components of the vector A, are specified by the input 
vector, and the last component coincides with the volume or the valuation of 
the output commodity. 

The production scheduling problem, which is the basis of the economic 
interpretation of linear-programming techniques, can also be written in 
canonical form, To this end we introduce m additional hypothetical produc- 
tion modes with unit input vectors (A,,,;=e,) and zero productivity evalua- 
tions. Under the m+i-th production mode only the é-th production factor 
is consumed (unit of é/-th factor in unit time) and nothing is produced. Hypo- 
thetical production modes are employed only in programs with excess of 
various production factors. 

The production program is specified by the vector X whose components 
give the times (intensities) of utilization of the various production modes. 
The optimal program ensures maximum valuation of the output commodity 
with given resources (or, equivalently, maximum production volume of 
the plant). 

Proceeding from the valuation of the output commodity, we may, ob- 
viously, evaluate (cost) the various production factors. The costing system 
Vir Mav sees Yq Of the production factors, i.e., the cost program, is defined 
as the solution of the problem dual with respect to the production schedul- 
ing problem. This problem entails minimizing the linear form 


io m 
L(Y) = 6.94 (3.4) 
=1 
subject to the conditions 
m 
Py tii > Cp J=1, 2, ..., a, (3.5) 
¥; 9, i=1, 2, @e ey m. (3 6) 


Linear form (3.4) is interpreted as the overall costing of the resources, 
Restraints (3.5) require that the cost of the overall input of the production 
factors under each production mode does not exceed the valuation of the 
commodity produced in the same time. Restraints (3.6) are trivial: the 
costs cannot be negative. 

The cost program specifies an optimal production program and, con- 
versely, an optimal production program corresponds to a cost program, 
i.e., a costing system of the production factors. We remind the reader that 
the production factors are valued in units of cost of the output commodity. 

Economically, the best production modes are those under which the value 
of the input is equal to the value of the output commodity. We shall refer 
to these methods as paying production methods. Nonpaying production 
methods are, obviously, deficit methods: the value of the output commodity 
produced under one of the nonpaying production modes is lower than the 
cost of the resources used. Nonpaying production modes do not utilize the 
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production possibilities in the most effective way possible. An optimal 
production program comprises paying production modes only. 

The preceding economic interpretation of the dual pair makes it possible 
to classify the finite linear-programming methods in terms of the produc- 
tion-scheduling problem. 

In the simplex method solution starts with some support program. 
Economically, this means that the solution of the problem starts with an 
analysis of some set of m nonnegative numbers x,, Xs,,...,%, which specify 
utilization intensities (times) for each of the m production modes with 
linearly independent input vectors A,,...,4,,. The production modes corres- 
ponding to the basis vectors should, obviously, be called the basis pro- 
duction modes. 

Writing the input vector A, in terms of the basis input vectors amounts 
to finding the times (intensities) x,,of utilization of the basis production 
modes for which the input of each of the production factors will equal the 
corresponding input under the j-th production mode in unit time, When 
the plant operates under the s,-th production mode for x,, units of time 
(i=1, 2,...,m) the utilization of all the basis production modes will ensure 


m 
an output of Dees units of commodity. The output commodity is valued in 
m =t 


PacaXy cost units. On the other hand, if the plant operates only under the 
=I 


j-th production mode in unit time, ¢, units of commodity will be released 
(or, equivalently, the production value will be ¢, cost units). Thus, given 
the same input of production factors, the volume and, consequently, the 


m 
value of the output commodity was >> C.,x,;, in the former case, and ¢, in 
=1 


the latter. The difference 
Ay= Leakiy—%y 


gives an evaluation of the j-th production mode with respect to the given 
system of basis production modes, The parameter A, shows the expediency 
of introducing the j-th production mode into the program, correspondingly 
reducing the utilization intensity of the basis production modes. If A,<0, 
this change is advisable. If A,>0, the change would reduce the production 
volume: the /-th production mode is less economic than the combination 

of the basis methods. 

Analysis of the initial production program thus amounts to computation 
of the parameters A, The transition to other basis production modes will 
not increase the output volume if A,20 for all 7, In this case, the program 
being analyzed is optimal. If some of the A, are negative, the production 
program can be improved by substituting the k-th production mode with 
A,<.0 for one of the basis methods, The method to be eliminated from the 
basis is chosen such that the new support program can be realized (x,>0 
for all /). 

The new program is again tested for optimality. After a finite number 
of program improvements, we arrive at an optimal schedule for the plant. 

We now give an economic interpretation of the dual simplex method, The 
solution of the problem starts with an analysis of a support program of the 
dual problem. A program of the dual problem can be interpreted as a 
system of preliminary valuations of the production factors. The system of 
preliminary valuations should satisfy restraints (3.5), (3.6). Moreover, 


457 


to a support program of the dual problem there correspond m paying (with 
respect to this program) production modes with linearly independent input 
vectors. We shall refer to the production modes as basis modes. Let 
the indices of the basis production modes be 5&,, S,,--.:5,. Production 
programs which comprise only paying production methods (with respect to 
the given system of preliminary valuations) need not be realizable, The 
times x, of utilization of the paying production modes should satisfy the 
conditions 


Les! 
b= 2 sXe A=1, 2, coe, M, (3.7) 
i=1 


If all x,,satisfying system (3.7) are nonnegative, the initial system of pre- 
liminary valuations proves to be the required ''cost vector'' and the corres- 
ponding production program is optimal. If some of the x, satisfying sys- 
tem (3.7) are negative, the production program corresponding to the pre- 
liminary valuation system of the production factors cannot be realized. The 
valuation system should, correspondingly, be improved, Tothis end, the r-th 
production mode corresponding to the negative x, is eliminated from among 
the basis modes. 

The choice of a paying production mode as the r-th mode was one of the 
reasons that the production schedule cannot be realized. The preliminary 
valuations should be refined by making the r-th mode nonpaying, and retain- 
ing paying methods in all the other basis positions. Increasing the deficit 


(A, = 2 aii c,) 


of the r-th mode, we should take care that the system y,, y,,...,y, still satis- 
fies restraints (3.5) and (3.6) of the preliminary valuations. The limiting 
system of preliminary valuations which we obtain in this procedure defines 
a new paying production mode which will be substituted for the r-th basis 
mode, The program comprising the new basis production modes should 
again be tested for realizability. The valuations are refined until the input 
vectors of paying (with respect to the current system of valuations) produc- 
tion modes define a realizable production program, 

The economic interpretation of the Hungarian method is best illustrated 
by one of the modifications of the method, which was called in Ghapter 7 
the method of bilateral evaluations, 

The solution procedure starts with an analysis of a system of 
preliminary valuations (a feasible program of the dual problem) and a 
production schedule (a support program of the primal problem), The given 
system of preliminary valuations specifies the total cost of the production 
factors used. The initial production schedule gives the total value of the 
output commodity. The given system of preliminary valuations and the 
initial production schedule are optimal, if the total cost of the production 
factors used is equal tothe total value of the output commodity. Otherwise, 
the deficit of the schedule is determined, i.e., the excess of costs over in- 
come, The method amounts to gradual reduction of the deficits, 

Applying only basis production modes corresponding to the initial support 
program and production modes which are paying with respect to the given 
system of preliminary valuations, we construct a production schedule mini- 
mizing the deficit of the preceding program. This schedule indicates the 
direction in which the system of preliminary valuations should be modified. 
If with modified valuations of the production factors the total value of the 
output commodity coincides with the production costs, the new production 
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schedule is optimal, and the corresponding system of preliminary valuations 
can be taken as the cost vector, Otherwise, another auxiliary problem is 
solved minimizing the deficit of the preceding schedule on the set of produc- 
tion modes whichare paying with respect to the corresponding system of pre- 
liminary valuations or belong to the basis methods of *:1e schedule in question. 
Successive modification of the preliminary valuations of production factors and 
of the system of basis production modes gradually reduces the deficit — the ex- 
cess of costs over income — and elicits the production modes and the cost system 
to be adopted with optimal results under the conditions of the problem in question. 

Other modifications of the Hungarian method can be given analogous eco- 
nomic interpretations. 

In the simplex method and in various modifications thereof, we are given 
an initial production program. An analysis of this program enables us to 
establish its optimality or to indicate a method for its improvement. For 
the optimal program we then determine the system of valuations of the pro- 
duction factors, the cost vector, 

In various modifications of the dual simplex method, the point of depar- 
ture is a given system of preliminary valuations of the production factors. 
The attempts to construct a production schedule corresponding to the sys- 
tem of preliminary valuations gradually refine the said valuations, The op- 
timal production program corresponds to the final valuation system. 

Finally, in various modifications of the Hungarian method, the process 
of solution starts with a production schedule and a system of preliminary 
valuations which, generally speaking, are improperly correlated andproduce 
a nonpaying program, i.e., excess of total costs over the total value of the 
output commodity. Successive reduction of the deficits produces an optimal 
production schedule and a final valuation system of the production factors. 

Economic interpretation of the linear-programming problem thus charac- 
terizes the principal finite methods as the best techniques for compiling a 
production schedule for a plant and valuing its resources. 

3-5. In conclusion, we consider some formal features of the linear-pro- 
gramming methods which can be used as a key in classification, 

Let the problem be given in canonical form. 

Maximize the linear form 


L=CX (3.8) 
subject to the conditions 
AX = B, (3.9) 
X>0. (3.10) 


Theorem 5. 2. proved in Chapter 3 makes it possible to replace the solu- 
tion of the extremum problem (3.8)—(3.10) by computation of an n-dimen- 
sional vector X and m-dimensional vector Y satisfying the following system 
of linear equations and linear inequalities 


AX = B, (3.11) 
X>0, (3.12) 
YA>SC, (3.13) 
CX = BY. (3.14) 


The last equality follows from the system of restraints 
(YA), =c, for x, #9. (3.15) 
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The four systems of equalities and inequalities (3.11)-(3.14) specify four 
groups of linear-programming methods. In each method, the transition be- 
tween successive iterations amounts to testing the vectors X and Y for 
which three out of the four restraint systems are satisfied, and the fourth 
system is taken as the optimality test of the pair (X,Y). 

Each iteration of the method, without violating the three restraint sys- 
tems, ensures a monotonic (in a certain sense) approach to the set of points 
(X, Y)on which the fourth system is satisfied. 

In the simplex method, the transition between successive iterations 
amounts to testing the support programs X of problem (3.8)—(3.10). Hence, 
restraints (3.11) and (3.12) are satisfied in each iteration, To each support 
program X there corresponds a unique vector Y satisfying restraints (3.15) 
and, consequently, equality (3.14), too, The simplex method thus amounts 
to testing the points (X, Y)on which restraints (3.11), (3.12), and (3.14) are 
satisfied, and restraint (3.13) is not satisfied. Inequalities (3.13) or, equi- 
valently, the restraints 


m m_ 
A= 2 aii Cy = Bis iy 0, 20 JaH13 Bikeds 3 


specify the optimality test of the method. 

Assume that in some iteration we have arrived at point (X,,Y,). The poly- 
hedral restraint set of the dual problem is defined by inequalities (3.13). 
Consider in the space of variables of the dual problem the hyperplane 


BY =CX,( =BY,). 


It can readily be shown (this is left to the reader) that each iteration of 
the simplex method monotonically reduces the distance of the hyperplane 
BY=CX, from the polyhedral set YA>C. The hyperplane corresponding to 
an optimal program is a support hyperplane for the domain of definition of 
the linear form of the dual problem, 

The transition between successive iterations in the dual simplex 

method corresponds to testing support programs of the dual problem. Hence, 
restraints (3.13) are satisfied in each iteration. Each support program Y 
of the dual problem specifies a pseudoprogram X of the primal problem, for 
which restraints (3.11) and (3.14) (a consequence of (3.15)) are satisfied. 
The dual simplex method thus involves testing the points (X, Y) which satis- 
fy restraints (3.11), (3.13), and (3.14). Restraints (3.12)— nonnegativity of 
the pseudoprogram components — serve as the optimality test of the pseudo- 
program. In each iteration the distance of the hyperplane 


CX = BY, (=CX,) 
from the polyhedral restraint set of the primal problem 
AX=B, X=>0 


is monotonically reduced. The hyperplane corresponding to the solution of 
problem (3.8)—(3.10) is a support hyperplane of the domain of definition of 
the primal problem. 

In the Hungarian method, to each iteration corresponds a program 
Y of the dual problem and the associated quasiprogram X of the primal 
problem. The vectors X and Y satisfy restraints (3.12), (3.13), and (3.14). 
Restraint (3.11) does not hold. The method amounts to testing the quasi- 
programs X (or, equivalently, the programs Y of the dual problem) which 
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. monotonically reduce the residual e, of the quasiprogram 


m 
&, =— = | (AX—B), . 


The computational procedure is simplified if, in addition, the residues 
are assumed to be nonnegative: 


@,=(B—AX),>0, i=1, 2,..., m. 


Restraint (3.11) (requirement of zero residues) is the optimality test of 
the quasiprogram., 

The methods of the fourth group imply testing the points (X, Y) corres- 
ponding to programs of a pair of dual problems. 

Thus, in each iteration of the method restraints (3.11), (3.12), and (3.13) 
are satisfied. Condition (3.14) is not observed. Each successive iteration 
monotonically reduces the difference BY—CX 

One of the methods of the fourth group amounts to simultaneous impro- 
vement of programs of the primal and the dual problems (see Chapter 6, 
8-3). The vectors X and Y represent support programs of the dual pair. 
Each iteration monotonically reduces the difference between the linear 
forms of the two problems. 

Another example of a method of the fourth group is the method of bi- 
lateral evaluations. Each iteration of the method produces a support pro- 
gram xX of the primal problem and a feasible program /Y of the dual prob- 
lem. The point (X, Y) satisfies restraints (3.11)~(3.13), and the difference 
BY—CX monotonically decreases with each successive iteration. 

In each of the preceding methods, we test support programs, pseudo- 
programs, or quasiprograms of the primal problem. The total number of 
Support programs, pseudoprograms, and quasiprograms of a problem is 
finite. Thus, for any of the preceding methods the total number of points 
(X, Y) satisfying three out of the four restraints (3.11)—(3.14) is finite. 
Moreover, in nondegenerate problems (we have seen that any linear-pro- 
gramming problem can be artificially reduced to a nondegenerate problem) 
the transition between successive iterations is connected with monotonic 
approach to the solution of the problem, These two points ensure that an 
optimal program will be attained in a finite number of iterations. 

Until now in this section we considered linear-programming problems 
in canonical form. The more common form of linear-programming prob- 
lems, where the restraint systems comprise both equalities and inequalities 
and the nonnegativity requirement is imposed only on some of the variables, 
makes it possible to classify linear-programming methods in more nu- 
merous groups, Instead of the two systems (3.11) and (3.12) defining the 
range of the linear form of the primal problem, in the general case we have 
three systems: 

A, X=B,; A,X>B; 
x,=0, JI day ven Sy: 


The number of various restraint systems defining the range of variables 
of the dual problem also increases (again three). Only restraints of type 
(3.15) remain unchanged. The solution of the linear-programming problem 
thus amounts to solving seven systems of linear equations and linear inequa- 
lities. In the new classification, each group comprises methods corres- 
ponding to testing the vectors (X, Y) satisfying six out of the seven restraint 
systems. The seventh restraint system is taken as the optimality test of 
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the vectors (X, Y). Each iteration of the method ensures a monotonic (in 
some sense) approach to optimal programs of the dual pair. 

We should emphasize that the characteristics of the various methods 
given in this article can also be applied for classifying iterative methods of 
linear programming. However, in this case, the set of points (X, Y) tested 
is infinite. Iterative methods will therefore produce an approximate (with 
any predetermined degree of accuracy) solution of the problem. 

The formal classification of the linear-programming method presented 
in this article also gives us an assurance of a uniform approach to all the 
methods used in solving nonlinear conditional-extremum problems and it 
can therefore be used in devising a general system of classification of 
mathematical-programming methods. 


EXERCISES TO CHAPTER 8 


1, Prove the equivalence of the definition of a support program in 1-1 and of the general definition given 
in Chapter 2, 4-3. 

2, Prove the equivalence of the definition of a nondegenerate support program in 1-1 and of the corres- 
ponding general definition stated in Chapter 2, 4-7. 

3, Prove that for any nondegenerate problem systems (1.27) and (1.28) each have one minimum, 

4, Prove the recurrence formulas (1,41), 

5. Applying the arguments of 1-6, 1-8, devise a simplex computational procedure for problem (1, 1)-(1.4). 

6, Draw a block diagram of the simplex algorithm for the case when several vectors are introduced 
simultaneously into the basis. 

1. Draw a block diagram of the dual simplex algorithm for the case when several vectors are eliminated 
simultaneously from the basis. 

8. If the linear form of the primal problem is bounded above and below in the domain of its definition, 
the polyhedral restraint set of the dual problem is unbounded. Prove. 

9, Linear forms of conjugate problems bilaterally bounded on the sets of feasible programs of these prob- 
lems retain a constant value in the domain of their definition, Prove. 

10, Devise a computational procedure for solving problem (1.47)-(1.50) by the simplex method without 
modifying the original form of the problem, 

11, Give an economic interpretation of the solution procedure of problem (1.47)-(1.50) by the simplex 
method, 

12, Devise a general dual simplex technique for linear- programming problems in arbitrary form. 
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APPENDIX 
MATHEMATICAL PRINCIPLES OF LINEAR PROGRAMMING 


We give here some elements of linear algebra and the theory of convex 
sets in finite-dimensional spaces, which constitute the mathematical ap- 
paratus of the theory of linear programming. 

In the first section we give the prerequisites relating to finite-dimen- 
sional vector spaces,.matrices, and determinants. This section is intended 
for reference purposes, all propositions are given without proof. The 
corresponding details can be found, e.g., in /121/. The next section (§ 2) 
deals with systems of linear equations. Particular attention is given to the 
geometrical interpretation of the propositions. In particular, in §2 we 
discuss the connection between the theory of systems of linear equations 
and the dual definition of a linear manifold. At the end of the section we 
describe one finite method for solving systems of linear equations (the 
Gauss-Jordan complete elimination method), which is particularly signifi- 
cant in linear programming. 

In the last section (§ 3) we give the necessary facts from the theory of 
convex sets in multidimensional spaces. We prove the theorem of sepa- 
rating hyperplane and comment on some important corollaries of this theo- 
rem, Moreover, in § 3 we give some useful propositions related to the 
concept of dimensionality of a convex set and to properties of its extreme 
points. 

All the propositions of § 2 and § 3 are proved. Exceptions are those 
which are left for the reader in the corresponding exercises. 


§ 1. Vectors, matrices, determinants 


1-1, An ordered system of a real numbers 
X= (X,, Xy, oe ey Ky) 


is called anaz-dimensional vector. The numbers x,,x,,...,*, are 
known as the components of the vector X. Vectors are denoted by 
upper-case letters, and their components by the corresponding lower-case 
letters. Two a-dimensional vectors 


X=(x,, Xe» eeey x,,) and Y=(y,, Ms: ees Yn) 


are said to be equal if their respective components are equal, i.e., if 


x,=y, for i=1,2,...,4. 
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The sum of the vectors X and Y is defined by the vector 
Z=(Z,, 2,, o00, Zq) 
whose components are z,;=x,+ y, #=1, 2, ..., — 
Z=XAV=(X, AIM, Mater oer Mata): 
Vector addition is commutative 
X+VY=VY+X 


and associative 


(X+VY)4+Z=X+(Y+2). 


The difference of an ordered pair of vectors X and Y is defined by 


vector Z 
LZ = X—YV=(X,— yy Fe— gn 0 or Lye Ip) 


A vector whose components are all zero is called the zero vector: 
O=(0, 0, ..., 9). 
For vectors X and Y to be equal it is thus necessary and sufficient that 
X—Y=0. 
The product of a vector X by a real number (scalar) a is defined by 
aX=(ax,, @X,, ..., %,). 
The properties of vector addition and scalar multiplication are: 
a, (a,X) =(a,a,) X (Associativity), 
(a, +a,)=a,X+a,X, 
a(X+ Y)=aX+eY, 
0X=a0=0. 


(Distributivity) 


The set of all a-dimensional vectors for which addition, subtraction, 
and scalar multiplication are defined as above constitutes a (real) a- 
dimensional vector space. The adjectiveinparenthesis real, in- 
dicates that the vector components and the scalars are real numbers, In 
the following this adjective will usually be omitted. This, however, should 
not lead to confusion, since we shall be dealing with real cases only. 

For a=3 thevector space defined corresponds to the ordinary: three-di- 
mensional space. To each ordered triad (x,, x,,x,) in this space corresponds 
a point with the coordinates x,, x,, x, (or a vector directed to this point from 
the origin). Conversely, to each point of the three-dimensional space cor- 
responds an ordered triad of real numbers consisting of the point coordi- 
nates. The elements of an az-dimensional vector space for a=8 (or na=2) 
can, thus, be interpreted either as points or as vectors directed from the 
origin of the coordinates. In this book the words vector and point are 
used synonymously for the elements of vector spaces, 

1-2. To every pair of vectors 


A=(a,, a,, ees a.) and B=(6,, b,, eeeys b,) 


there corresponds a real number 


(A, B)= Pu tidi 
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called the scalar product, and such that: 

1. (A, B)=(B, A). 

2. (A, +A,, B)=(A,, B)+(A,, B). 

3. (AA, B)=A(A, B). 

4. (A, A)>0, the equality applying only when A=0. 

3. (A, B)" => (A, A)(B, B). This inequality is generally attributed to Bunya- 
kovskii. 

Two vectors A and B are said to be orthogonal if 


(A, B)=0. 


In particular, the zero vector is orthogonal to any vector in the givenspace. 
We define the distance between points in an 2-dimensional space (specify 
the space metric) with the help of the scalar product, 
The length, or norm, of a vector 


A=(@,, @,, ..., @,) 


|AJ= +VA, 4= +V/ >» at. 


According to property 4 of the scalar product, |A|=0 for any vector A, |A|=0 
applying only when A=0. 
The distance between the points A and B is 


(4, B)=|A—Bi=J/ 3 (a;—5)". 


Let A, B, and C be arbitrary points in an a-dimensional space. Applying 
the properties of the scalar product, we prove the following inequality 
(the triangle inequality): 


|A—B| <|A—C]+|C—B]. 


is defined by 


The triangle inequality, like the Bunyakovskii inequality, has a clear 
geometrical meaning for n=2, 3. It is known that the scalar product of two 
vectors on the plane or in three-dimensional space is equal to the product 
of their lengths multiplied by the cosine of the angle between them. Hence, 
the above inequality can be rewritten in an equivalent form 


A, B 
rater 
which indicates that, inourcase, the cosineof the angle is less than or equal 
to unity. The triangle inequality is interpreted as follows: the sum of two 
sides of a triangle is no less than the length of the third side. 

An n-dimensional vector space in which the scalar product has been de- 
fined (and, hence, a metric has been specified) is generally called 
Euclidean. In the following an nz-dimensional Euclidean space will be 
denoted by &,. 

We say that a sequence of vectors X,, X,, ...,X,,...in &, converges to the 
vector X in E, (lim X,=X) if for any e>0 there exists a natural number AN(e) 


so that for all is> N(6) 
| X—X,|<e. 


It follows from the definition of the norm that the sequence {X,} converges 
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to X if and and only if 


lim x(*) = x,, s=t, 2, oo ey A 
kR->@ 
Here 
k k (k) 
ne ae Lodi ee) 3 ee ere 
A= (Kis Kh has): 


1-3. A rectangular array of numbers 


ai Gis ay © Qin 
a, Qs, a. o» =@,, 

A oo e ee &* @ ¢* e#* ¢ &¢® @& &#® 68 & @ : 1 1 
Qi, aig Qi, Qin ( ) 


Que eee any eae Can 


arranged in m rows and az columns is calleda mxn matrix. 

The entries in matrix A are called the elements of this matrix. 
Each element has an ordered pair of subscripts. The first subscript indi- 
cates the row number, and the second the column number; the intersection 
of this row and this column is the given element. For example, the ele- 
ment a,, is the intersection of !-th row and /-th column of matrix A. 

The matrix (1.1) is generally written more concisely, thus: 


A=[laylle, a: (1.2) 


If the dimension of the matrix A is given in the text, the subscripts m and 
nin (1.2) may be omitted. The matrix A is sometimes written as 


A = (A,, Ay eooeg A,,)s 
where A, is the j-th column of matrix A, or 
A —= (A™, A®), eee, A™), 


where A( is the i-th row of matrix A. 

Matrix A is called square if m=n and is said to be of ordera. In 
concise notations, the order of a square matrix is generally indicated by a 
subscript on the right of the double vertical bar: 


Hass, 


The elements a,, i=1, 2,...,a of the square matrix |/a@,,{|, constitute the main 
diagonal. 

If the matrix A has only one row or one column (m=tor a=1), it may 
be considered as a vector. In the first case, 


A= (O14) Bygs oo ey Bn) 


— an rz-dimensional row vector, and in the latter — 


— an m-dimensional column vector. 
We next define the operations of matrix addition and multiplication. 
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The sum of any two matrices A=||a,,||,,, and B= |l ,\l,,, iS given by the 


matrix 


Ryn 
C= [liz llean = Nl iz t Oy Mean 


Any element of the matrix C=A+8 is equal to the sum of the corresponding 
elements of the matrices A and B. Addition is defined only for matrices 
with the same dimensions. 

Consider the two matrices, S=||s,;\l,,, and T=([lt,ll,,,. The number of 
columns of S is equal to the number of rows of 7 The product of 
matrix S by matrix 7 is given by 

D=ST=||4islajn 
where 


k 
djj= Desay. (1.3) 


Multiplication is defined only on the assumption that the number of columns in 
S is equal to the number of rows of 8. The element in the i-th row and 
j-th column of matrix D (the product matrix of S and 7) is equal to the sum 
of the products of the elements of the i-th row of S multiplied by the cor- 
responding elements of the j-th column of 7, 

Matrix addition and matrix multiplication have the following properties, 
and are often used in matrix algebra: 


1. iy Mayn tl Bis Mene = Oy Win +l iy Mwy 
(Commutativity of addition), 
2. MW egy Mh aay EU By eam Ed Cay Ul coy) = (US iy Mey FU Bey nye) Fl Cry Wey 
(Associativity of addition), 
BS. (Ney Nase Oe 5 Maes) Wey Mey = Sey Maw, ao (UN Oey Maye Ul ery ley) 
(Associativity of multiplication), 
Fe Mey Mane (MSs Mee M Os ghey) =MNEr ln alle ley les lon, ll; lleesni 
(Ma; Manse H MD; lle, dM Er pl ey = Me lm, alles sles MO; lb, ells lesen 
(Distributivity of addition and multiplication), 
It is easily verified that matrix multiplication is not commutative, i.e., 


ey Meyer y Wiese FM Oey Mrs Ul Sey Mey re 


We shall usually deal with square matrices. We note that square matrices 
can be added and multiplied only if they are of the same order. Among 
square matrices of order n we have the unit (identity) matrix 


100 .w.. 9O 


os 
_o°o 
(oo) 
—_ © 
Oo ©} 


00 0... 1 
whose main diagonal elements are 1, and whose off-diagonal elements are 
all equal to zero. For any matrix A of order a 

Al, =1,A=A, 


Like vectors, matrices may be multiplied by scalars. The product 


467 


of a matrix A=|l¢,,{|,,, by a scalar a is given by 
B=aA=|( 44; ,\lnin 


Thus, multiplication of a matrix by some number implies that each element 
of the matrix is multiplied by this number. We bring another operation 
carried out on a single matrix. 

The transposed matrix with respect to matrix (1.1) (or the 
transpose of matrix (1.1)) is given by the matrix 


a, a, eee ai, di Gar 
ais a,, ais ee.8 Ons 
Al Sm | (Oca : 
Gi, %%, Giz > Ons 
ain a, coe Gin vee Gan 


whose columns are the corresponding rows of the initial matrix A, Trans- 
position of a matrix A is denoted by the superscript 7. In particular, the 


column vector 


a,, 


Gaim 
may be written as 


A =(a,,) G53) eoeey Aim)'s 


Transposition has the following properties: 

1, (A+B)? =AT-+ BT, 

2. (AB)? = BTA’, 
i.e., the transpose of a sum is equal to the sum of transposed summands; 
the transpose of a product is equal to the product of transposed factors 
taken in reverse order, 

1-4, Any set of nm numbers 1, 2, ..., a arranged in some definite order is 
called a permutation of a numbers (symbols), The number of permu- 
tations of nm numbers is equal to a! 

We generally say that the numbers i and / constitute an inversion in 
the given permutation if i>j and still i precedes / in this permutation. 

A permutation is said to be even, if its symbols constitute an even 
number of inversions, and odd otherwise. The number of inversions in 
a given permutation (and consequently its parity) can be determined in the 
following way. 

Consider a permutation /=(i,, 1, ..., é,). Let k, be the number of symbols 
smaller than /, which are right of the symbol /,. The number of inversions 


of permutation / is equal to 
kth, +... +8,-) 


For example, the number of inversions in the permutation (3, 1, 4, 2, 6, 5) 
of six symbols is equal to 
24+0+1+0+1=4., 


This is therefore an even permutation. 


These preliminary facts form an introduction to the concept of the deter- 
minant of an arbitrary square matrix. 
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Consider the square matrix 


ais 18 ene ® Gin 
7 |e (79 (1.4) 
a a a 


Let us now write out all possible products of na elements of the matrix A in 
each of which there appears one and only one element from each row and 
each column, Each of these products can be written in the form 


Qs), a3}, eee Ganja (1.5) 
By definition, the subscripts j/,, /,, ..., /, are all different and consequently 
constitute a permutation 

Us Wis eeeys In) (1.6) 


The determinant of matrix A is obtained as the sum of all possible 
products (1.5). Each product has the sign (—1)* where d is 0 or 1 depending 
on whether the permutation (1.6) is even or odd, Each determinant has al! 
such products. 

The determinant of matrix A is generally denoted by |A|. Other nota- 
tions are sometimes used, corresponding to the various written forms of 
the matrix A: 


a. a,, eee 
|A]= a. a. eoe as, =Il(A A A —_ AM A”) (m) 
7 eee ow ee =| Mal = ay “v2? fees n) | =| ( ’ peers A )|. 
ans Ans P18; aun 


The elements, the rows, and the columns of matrix A are generally also 
the elements, the rows, and the columns of the determinant |A|. 

The following properties of determinants are often used in computations. 

1, |A|=]A7|, i.e., the determinant of a matrix is not affected by trans- 
position. 

2. If every element of a row (or of a column) of a determinant is zero, 
the value of the determinant is zero. 

3. If two rows (two columns) of a determinant are interchanged, the sign 
is changed but not the value of the determinant. 

4, A determinant containing two identical rows (two identical columns) 
has a value of zero. 

5. If every element of a row (column) of a determinant is multiplied by 
a number a, the value of the determinant is multiplied by a. 

GB. [(AAgy oo 6p AP AD «00s And [= LAgs Age oe Ajs eer ADIEU (Ags Age oer Aly oe ee Aad f 
(AM, AGM, 20.6, AO 4AM, ..., Al) | =] (AM, AM), .., AM, 22, AMM) [+] (AM, A®), 2.2, AM, ..., AMY, 


The first relationship can be stated in words as follows. If the j-th column 
A, of a matrix A is equal to the sum of two vectors A, and A), the deter- 
minant |A| is equal to the sum of the determinants of matrices A’ and A” in 
which all the columns, except for the j-th, are the same as in matrix A, 
and the /-th column of A’ (A") is A,(Aj). The second relationship is inter- 
preted analogously, writing rows instead of columns, 
7. The value of a determinant is not changed if to one of the rows (co- 

lumns) we add another row (column) multiplied by an arbitrary number, 
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8. Let a matrix A of the form (1.4) be given. In A cross out the i-th 
row and the j-th column, whose intersection is the element a, The deter- 
minant of the new matrix obtained, of order ra—1, is denoted by M,,. 

The cofactor of the element a,, of matrix A is (—1)'7M,=A,. 

The following equalities apply: 


ia 
[A] = a, Ai; 
nt 


|A|= me risAy-: 


In the first case we have an expansion of |A[ in the i-th row; in the second 
case, the expansion in the j-th column. 
9. Let A and B be square matrices of the same order. 


In this case 
| AB} =| Al] By}, 


i.e., the determinant of a product of two Square matrices is equal to the 
product of the determinants of these matrices. 

1-5. A square matrix A is called regular if its determinant, |A], 
does not equal zero, If |A|=0 the matrix A is singular. 

Let A be a regular matrix of form (1.4). Consider the matrix {[a,,|l, 
whose elements a,, are given by 


Gy =a, i, jJ=1, 2, coe, M, (1.7) 


Here A,, is the cofactor of the element a, of matrix A. Applying the fourth 
and the eighth properties of determinants, and also the definition of matrix 
multiplication, we obtain 


Alla, |,=Weyl,A=M (1.8) 


where J, is the unit matrix of order zn. We leave it to the reader to verify 
equality (1.8) (see Exercise 2). 

It can be shown (see Exercise 3) that the matrix Wisin is the only matrix 
satisfying (1.8). 

A matrix A™' is called the inverse (or the reciprocal matrix) 
of (or with respect to) matrix A if 


AA“'=ATA=/,, 


Thus, for a regular matrix A 
A~'= |] a, ,[l,5 


where the elements Gi, are given by (1.7). 

If the matrix A is singular it has no inverse. The proof of this is left 
to the reader (see Exercise 4). 

1-6. Let A,, A,,...,A, be an arbitrary set of n-dimensional vectors. 


The vector 
a,A,+a,A,+...+4,A,, 


where a, is a Scalar (/=1, 2,...,s)is calleda linear combination of 
vectors A,, A,,...,A, with the coefficients a,, a,,...,a,. The system A,,A,,...,A 
is said to be linearly dependent if the zero vector 
0=(0, 0,...,0) 
— ee” 


s 


2154 
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canbe represented as a linear combination of the vectorsA,, A,,...,A,in which 
some of the coefficients are nonzeroes. Otherwise the system A,, A,,...,Ay, 
is linearly independent. In other words, a system A,, A,,...,A, is said to be 
linearly independent if 

LJ 

t+ | 


implies that a4,.=a,=...=a,=0. Vectors constituting a linearly independent 
system are said to be linearly independent. 

It can easily be verified that the representation of a given vector as a 
linear combination of linearly independent vectors is unique. 

We now give a criterion for establishing linear independence of a system 
of vectors; this criterion is widely used in various branches of mathematics. 
For a system of vectors A;=(4,, @,,....4,), f=1, 2,...,s to be 
linearly independent it is sufficient and necessary that 

there be a square matrix of orders, 


Bij, Byjg nee By, 

Bsf, Tegfp oes Asi, 
eseetesvnee teeeea y 

Gi, Gs}, coe Us], 


whose elements are the components of vectors A;, such 
that its determinant does not vanish. 

If, in particular, s=a, for the given system to be linearly independent 
it is necessary and sufficient that 


\(A,, A, wey Ay) | 0. 


It follows that in an a-dimensional space there cannot exist a linearly inde- 
pendent system of more than az vectors. The unit vectors e,=(1, 0, 0,...,0), 
e,=(0, 1, 0,...,0),...,¢,=(0, 0, 0,...,1) constitute a linearly independent 
system. 

Analogously to a plane and a three-dimensional space, the dimen- 
sionality of a vector space is defined as the maximum number of linearly 
independent vectors in this space, Thus, the dimension of an a-dimen- 
sional vector space is a. 

Let us now consider an arbitrary rectangular matrix A of the form (1.1). 

A matrix comprising elements of the intersection of some rows and 
columns of A is called a submatrix of matrix A. The determinant of 
any Square submatrix of the matrix A is called the minor of this matrix. 
The order of a minor is determined by the order of the corresponding sub- 
matrix. 

The largest square array in A whose determinant does not vanish is 
called the rank of this matrix. 

It follows from the criterion of linear independence of a vector system 
that the maximum number of linearly independent rows of amatrix A is 
equal to the maximum number of linearly independent columns of this 
matrix, both coinciding with the rank of A. The proof of this is left to the 
reader (see Exercise 6). 

The rank of an arbitrary system of vectors is defined as the 
maximum number of linearly independent vectors in this system. The set 
of all linearly independent vectors of a system, whose number is equal to 
the rank of the system, forms the basis of the system. A set of linearly 
independent vectors of a given system constitutes a basis if and only if any 
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vector of the system can be represented as a linear combination of vectors 
of the set being considered. Necessity isobvious, Sufficiency can be proved 
proceeding from the criterion of linear independence of a system of vectors, 

1-7. In the analysis of systems of linear equations, which is the subject 
of the next section, we shall use properties of some important sets of points 
in an a-dimensional space. 

A nonempty set of a-dimensional vectors will be called a subspace of 
an n-dimensional vector space if the sum of any two vectors of the set and 
the product of any vector of the set by a scalar belong to this set. 

All subspaces, obviously, contain the zero vector, since 0-X=0 for any 
element X of the subspace. 

A subspace is said to be r-dimensional if the maximum number of 
linearly independent vectors in it is equal to r. Let A,, A,,...,A, be a syS- 
tem of linearly independent vectors in an a-dimensional vector space. Con- 
sider the set of all possible linear combinations of this system. It can 
easily be shown that it constitutes an r-dimensional subspace. On the other 
hand, in any r-dimensional subspace there exist r linearly independent 
vectors whose linear combinations span the subspace. The proof of these 
propositions is left to the reader (Exercise 7). 

Thus, an r-dimensional subspace can be defined as the set of all linear 
combinations of some linearly independent system of r vectors. If r=0, 
the subspace consists of the zero vector only. If r=a, it coincides with the 
entire space. If 0<¢<r<ian we have intermediate subspaces containing an in- 
finity of elements and not coinciding with the entire space. 

The set of elements of an a-dimensional vector space which can be writ- 


ten in the form 
X+X, 


where X is a fixed vector, and X’ belongs to some r-dimensional subspace, 
is called an r-dimensional linear manifold (rsa), 

Thus, an r-dimensional linear manifold may be considered as some 
translation of an r-dimensional subspace. In particular, if the translation 
(the vector X) is the zero vector, the manifold reduces to the subspace. 

A one-dimensional linear manifold will becalled a line, andan(a—l)- 
dimensional linear manifold, a hyperplane. 

Remembering the definition of an r-dimensional subspace, we ascertain 
that an r-dimensional linear manifold is actually a set of points of the form 


X+ BaX,, 
where X is a fixed point in the n-dimensional space; X,, X,,...,X, is some 


linearly independent system of n-dimensional vectors, anda,, a,...,a, are 
arbitrary numbers. 


§ 2. Systems of linear equations 


2-1, Consider the system of linear algebraic equations 


a 
2 GX ,=5;, i=1, 2, ooey M, (2.1) 
=i 

comprising m equations in a unknowns x, (j=1,2,...,2). 
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The matrix A=||a,,||,,, whose elements are the coefficients of system 
(2.1) is called the matrix of coefficients of system (2.1). If we take 
A,=(@,, @,,, ..-, @,,)7 the matrix of coefficients of (2.1) can be written in the 


form 
A=(A,, A,, «..; A,). 


B=(b,, by, «++ 0,)7. 


The vector B is called the free-term column of the system, 
System (2.1) may be rewritten in the following equivalent forms: 


> A x ,=B; (2.2) 
AX=B, (2.3) 


where X=(x,, 4) 0.) X,)7. 

The matrix A=(A,, A,, ..., A,B) is called the augmented matrix of 
system (2.1). The vector X’=(x,, x, ...,%,) is called a solution of system 
(2.1) if substitution of x, for x; (j=1, 2, ..., a) in the system reduces all the 
equations of (2.1) to identities. 

A system of linear equations which has at least one solution is said to be 
consistent. 

If the system has no solutions, it is said to be inconsistent, orcon- 
tradictory. Acconsistent system is sometimes called solvable, and 
an inconsistent one, unsolvable. 

In the following we give conditions for consistency of systems of linear 
equations and describe the characteristics of sets of solutions of consistent 
systems. Moreover, we present a method for solution of linear equations 
which is widely used in linear programming. 

2-2. We shall start our discussion with the case in which the number m 
of equations in system (2.1) is equal to the number a of unknowns. The 
matrix of coefficients A is then a square matrix of order a, 

Let us assume that the determinant of matrix A does not vanish. Let 
the vector X’=(x,, x, ..., Xx)" be some solution of system (2.1). Using (2.3), 


btai 
we o in AX’ =B. 


By multiplying both parts of this equation from the left by the matrix A™', 
which is the inverse of the matrix A of the system (the existence of A™ foi- 
lows from the condition |A|#0), we obtain the following chain of equations: 

X' = [X' = (A~"A) X7 = Aq'B. (2.4) 
Thus, any solution of system (2.1) can be represented by (2.4). If the sys- 
tem (2.1) with square matrix A is consistent, it has a unique solution. 

We shall show that the vector X’=A™-'B solves the system (2.1). Indeed, 


AX’ = A(A7'B)=(AA~') B= /B=B. 


The system (2.1) with n=m and |A|#0 is, thus, consistent and has a unique 
solution determined by (2.4). 

Let us now transform (2.4). Applying (1.7), which defines the elements 
a;, of the inverse matrix A“', andthe rule of matrix multiplication, we obtain 


R n 
, ~ A 
£)= Dy Obs Dah by. (2.5) 
s=1 8=1 
According to the eighth property of determinants, we have 


2 A= MAy A, eee, Aye B, Ay+y eee A,)|- (2.6) 
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From (2.5) and (2.6) we have 


, 


gial(An A 


[dy Ay soos Aa) aa oo 
which is known as Cramer's rule, and can be stated as follows: 

If the square matrix A ofasystemof linear equations is regular, the j-th 
component of the solution of the system is equal to the determinant of a 
matrix which is obtained from A when the /-thcolumn is replaced by the 
free-term column of the system divided by the determinant of matrix A. 

2-3. We now deal with the general case in which no restrictions are im- 
posed on the matrix of coefficients of the system of linear equations (2.1), 

Let the rank of matrix A be r. We disregard the trivial case r=0, i.e., 
the matrix A consists of zeroes only. The basis of the system A,, A,, ..., A, 
of vectors constituting the matrix A contains r vectors. To be specific 
let us assume that the vectors A,, A,, ..., A, form the basis. Further, as- 
sume that the system (2.1) is consistent and that X=(x,, x,, ...,x,) is one of 
its solutions. In this case the free-term vector B is a linear combination 
of the vectors A,, j=1, 2, ..., a: : 

=2 x A; 


Since each of the vectors A, /2r+1, can be expressed in terms of the basis 


vector, we have 
r 


B= 2 aA). 

Hence, the vectors A,, A,, ..., A, constitute a basis of system A,, A,, ...,A,, 8B 
and, consequently, the rank of the augmented matrix A=(A,, Ay, easy Agy 1B) 
of system (2.1) is also r (the rank of matrix 4A). 

Now let the ranks of matrices A and A coincide at the outset, both being 
equal to r. 

If A,, A,,..., A, 1S a basis of the system A,,A,,...,A,, there exist numbers 
a, (i= 1, 2, ..., 7) Such that 


B=> a,A;. (2.8) 


Otherwise the system of vectors A,, A,, ..., A,, B would be linearly indepen- 
dent and the rank of matrix A would be greater than r. 

Equality (2.8) indicates that the vector (a,, a,, ..., @,, 9,..., 0) solves the 
system (2.1). 

We have thus obtained the following criterion for the solvability of a sys- 
tem of linear equations. 

Theorem 2.1. Fora system of linear equations to be consistent, it is 
necessary and sufficient that the rank of the coefficient matrix equal the 
rank of the augmented matrix. ) 

It is noteworthy that Theorem 2.1 applies also for r=0, since in this 
case the system is solvable if and only if B=0. Hence, the matrix A and 
the matrix A both have rank zero. 

Having established the criterion for solvability of a system of linear 
equation, we now must deal with the problem of the complete set of its 
solutions. 
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The system of equations 


Ma 


G,,x=0, j=l, 2, 0.) m, (2.9) 


~, 


=1 


or, equivalently, 
AX=0 (2.10) 


is called homogeneous, i.e., the free-term vector is zero. The following 
forms of system (2.9) are convenient for applications 


>» A ,x,;=9, (2.11) 
(AM, X)=0, t=, 2, ..., ™ (2.12) 


Here A=(a,,, aj, «++; @,) is the ¢-th row of matrix A. 

Since the matrix of coefficients of (2.1) and of (2.9) are identical, the 
system (2.9) is said to be the homogeneous system corresponding to 
system (2.1). 

Let X, and X, be two solutions of system (2.1) and Y=X,—X,. Obviously, 


AY=AX,—AX,=0. 


Hence, the vector Y is a solution of the homogeneous system (2.9) (or (2.10)). 
On the other hand, if Y is a solution of system (2.9), and xX, is a solution 
of system (2.1), then X,=X,+Y is also a solution of system (2.1), since 


AX, = AX, + AY=B+0=B, 


Hence, the set of all solutions (the general solution) of system (2.1) 
can always be represented as the sum of a particular solution of this sys- 
tem and the general solution of the corresponding homogeneous system (2.9). 
2-4. We shall now investigate the general solution of the homogeneous 
system (2.9). 
Consistency of system (2.9) is obvious, since the zero vector 0= (0, 
0, ..., 0) is always a solution. Let the rank of the matrix of coefficients in 
(2.9) be r. This is equivalent to saying that the maximum order of a non- 
vanishing minor of matrix A is r. To be specific, let |||a,,||,|40. In this 
case the first r rows of matrix A are linearly independent. 
We shall say that two systems of linear equations in a unknowns are 
equivalent if the general solutions of the two systems coincide. 
Consider the system of equations 


Pa 21p%;=1 i=l, 2, eee lr, (2.13) 


consisting of the first r equations of system (2.9). Using (2.12), we re- 
write (2.13) in the form 


(AM, X) =0, i=1, 2, eoes r, (2.14) 
We have 
r 
AO = & aA", s=r+l, coe, M, 
a1 


Hence, if the vector X satisfies system (2.14), then 


(AM, H=d a,,(A%, X)=0,  s=r-+l, oo.) m, 
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We observe that systems (2.9) and (2.13) are equivalent, i.e., the general 
solution of one is also the general solution of the other. 
We rewrite system (2.13) in the form: 


r n 
> ayxj=— 2 aj%p 1,2) exay fF: (2.15) 
By assumption, the determinant of the matrix ||a;,\|, does not vanish. Hence, 
for any values of the unknowns Xp J=Hrtl, ..., 0 the first r unknowns of sys- 
tem (2.13) are uniquely determined; moreover, they can be calculated by 
Cramer's rule. 

Let us introduce a system of 2-dimensional vectors 


Mee Re, ews (0s gag Oe 10x sans O); 
B 
where the numbers «!", xf”, ..., x! are determined from the system 
r 
2 4p! =— 4, eee. Lely 2 acct; (2.16) 


and the superscript s takes on the values from 1 to a—r. 

Since the solution of system (2.15) is unique for any fixed value of its 
right-hand side, we obtain the following expression for an arbitrary solution 
X' =%,, Xp 200, Xn) Of system (2.13): 


Ror 


X = > x7 42X,. (2.17) 


On the other hand, since (2.13) is a linear and homogeneous system, the 


nf 
vector }) a,X, solves this system for any a,, a,, ...) @,-5. 


g=1 

The general solution of system (2.13) (and consequently also of the equi- 
valent system (2.9))is the set of all possible linear combinations of the vec- 
tors X,,X,,...,X,.,.. The last n—r components of the vectors X,, s=1, 2, ...,a—F 
constitute a square matrix with a nonvanishing determinant. The vectors 
X,, X,,..., X,_, are, thus, linearly independent. 

We have proved the following proposition: 

Theorem 2.2. The set of all solutions (the general solution) of a homo- 
geneous system of linear equations in n unknowns whose matrix of coeffi- 
cients is of rank r constitutes an (n—r)-dimensional subspace (the solution 
subspace). 

2-5. Let the system (2.1) be consistent, its matrix be of rank r, and X 
a solution of the system. Applying the previous relationship between the 
general solutions of systems (2.1) and (2.9) we ascertain that linearly inde- 
pendent vectors X,, s=1, 2, ..., a—r (these are solutions of system (2.9)) 
exist such that the set of all vectors : 


a—f 


X+ 3% 4,X, 


for various values of the coefficients a,, a,, ...,a,._, coincides with the 
general solution of system (2.1). 

The assertion with regards to the general solution of system (2.1) can be 
summarized in the following theorem: 
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Theorem 2.3. The general solution of a consistent system of linear 
equation in n unknowns having a matrix of coefficients of rank r is an (n—r)- 
dimensional manifold. This manifold is obtained by translating the solution 
subspace of the corresponding homogeneous system by any vector satisfying 
the initial system. 

The uniqueness of the solution of a system of linear equation follows as 
a corollary from Theorem 2.3. 

Theorem 2.4. A solvable system of linear equations has a unique solu- 
tion if and only if the rank of the matrix of coefficients is equal to the num- 
ber of unknowns. 

In particular, if the number of equations and the number of unknowns are 
equal, a necessary and sufficient condition for uniqueness of solution is that 
the matrix of coefficients be regular. 

Consider an arbitrary e-dimensional linear manifold comprising the 
points 


e 
X,+ 2X, (2.18) 
where X,, X,, ..., X, is a linearly independent system of vectors (g<n). We 
shall use the vectors X,, X,, ... X, to construct a homogeneous system of 
equations 
(X,, Y)=0, $221); Zone 1@ (2.19) 
The components of the vector Y=(y,, y,, ..., y,) are the unknowns of the sys- 


tem (2.19). The rank of the matrix of coefficients of system (2.19) is a. 
According to Theorem 2.2 the general solution of the homogeneous system 
(2.19) can be written as 


a,Y,+aY,+...+4,_,Y,_, 


where Y,, ¢=1, 2, ..., a—@ are linearly independent vectors, and 4q,, a,, ..., a,_, 
are arbitrary numbers. In particular, 
(X,Y )=0 

for ix1, 2, ...,Q; j=1, 2, ..., a—@Q. Hence, the homogeneous system of equa- 
tions 

(Y,, X)=0, i=1, 2, ..., 2—Q, (2.20) 
is solved by the vectors 

Xi Xap vay Me 
Since the vectors Y,, Y,, ..., ¥,., are linearly independent, we conclude 


that system (2.20) is of rank a—e. Therefore, according to Theorem 2.2, 
the general solution of this system is a g-dimensional subspace. The vec- 


tors X,, X,,...,X, are linearly independent and are contained in this subspace. 
Hence the general solution of system (2.20) has the form 
a,X,+0,X,+...+4,X,. (2.21) 


We now introduce the following system of linear equations: 
(Y;, X)=(Y;, X,), i=1, 2, ..., A—Q. (2.22) 


Obviously, X, is one of the solutions of system (2.22). Theorem 2.3 and 
(2.21) which represent the general solution of system (2.20) show that the 
expression (2.18) is the general solution of system (2.22). 
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We have thus established the converse of Theorem 2.3. 

Theorem 2.5. Anarbitrary q-dimensional linear manifold represents 
the general solution of some system of linear equations in n unknowns with 
matrix of coefficients of rank n—g. 

Consider one linear equation in a unknowns: 


n 


Pa 2)%)= 0. (2.23) 


Let the vector A=(4,, @,, .-., @,) be any but the zero vector, It follows from 
Theorem 2.3 that the general solution of equation (2.23) is a (n—1)-dimen- 
sional linear manifold, i.e., a hyperplane. 

On the other hand, according to Theorem 2.5 any hyperplane is a general 
solution of some equation of the form (2.23) with vector A=(a,, a,, ..., @,)#0. 
From these considerations we obtain another definition of a hyperplane. 

A hyperplane is a set of points in an ra-dimensional space satisfying 
a linear equation of the form (2.23), where A=(a,, a,, ..., a,)#0. 

Equation (2.23) is the equation of a hyperplane. The vector 
A#0 is called the direction vector of the hyperplane specified by (2.23). 

Let X, be an arbitrary point on the hyperplane (2,23) (the hyperplane de- 
fined by (2.23)). Hyperplane (2.23) may, obviously, be considered as the 
set of all points which can be represented in the form X,+X’, where (A, X’)=0. 
The hyperplane (2.23) is, thus, a translation, by X,, of the set of vectors 
X’ orthogonal to the direction vector A. Therefore we can say that a hyper- 
plane is orthogonal to its direction vector. 

In discussing linear equations, we may always assume that not all the 
coefficients of each equation are zero. Indeed, if the system contains an 


equation 
OX, +0X,+...+0X, =), 


then for 6#0 the system is inconsistent, and for }=0 this equation does not 
impose any restraints on the vector X andcan, therefore, be omitted. 

Hence, the general solution of a consistent system of linear equations 
is the intersection (the common area) of several hyperplanes (depending on 
the number of equations in the system). 

We shall say that hyperplanes are linearly independent if their 
direction vectors constitute a linearly independent system. 

Using Theorems 2.3 and 2.5 we can define a linear manifold in a different 
manner. 

The intersection of r linearly independent hyperplanes (provided it is 
nonempty) is called an (2z—r)-dimensional linear manifold. 

An (a—r)-dimensional linear manifold may, thus, be considered 
either as a translated set of linear combinations of (a—r) linearly inde- 
pendent vectors, or as the intersection of r linearly independent hyper- 
planes. 

The proof of the equivalence of the two definitions following from Theo- 
rems 2.3 and 2.5 is dealt with in the theory of linear equations. 

In Chapter 3 analogs of Theorems 2.3 and 2.5 were established with re- 
gards to systems of linear equalities and inequalities. These propositions 
make it possible to prove the equivalence of the two different definitions of 
a convex polyhedral set— the range of the linear form of the linear-pro- 
gramming problem — which is a generalization of the linear manifold. 

2-6. We give now another criterion of consistency of systems of linear 
equations which in some cases proves to be more convenient than Theorem 2.1. 
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Theorem 2.6. Fora system of linear equations (2.1) to be consistent it 
ts necessary and sufficient that the equality 


be satisfied for any vector Y=(y,, y,, .-.. Ya) Satisfying the homogeneous 
system 
(A, N= 2 4,,;=0, jJ=1, 2, CO ey A, (2.25) 


The proof proceeds from the following lemma. 
Lemma 2.1. Let X,, X,, ..., X, be an arbitrary linearly independent 
system of vectors. The matrix 


(X,, x,) (X,, X,).. (X,, X;,) 
ve (X,, X,) (Xy, X,).. (X_, Xy) ; 
(X,, X,) (Xg, X).--(Xg, Xy) 


whose elements are the scalar products of these vectors, is regular. 
Proof. Consider a homogeneous system of equations with the matrix of 


coefficients M: 
h 


»2 (X;, X)yj=0, t= 1, 2,..., 2%, (2.26) 
=1 
Let A=(A,, 4,, ..., 44) be an arbitrary solution of system (2.26): 

a 

2 (Xp X)0,=0, t=, 2,..., 2%, (2.27) 


Using properties 2 and 3 of the scalar product we rewrite equations (2.27) 
in the form 


(X, R)=0, t=, 2, 2.0, & (2.28) 


where 


Multiplying the i-th equation of system (2.28) by 4, and summing the results, 
we obtain 


k k 

D> A(X, R= (2 A,X, R)=(R, R) =0. (2.29) 
Equation (2.29) indicates that 

h 
R= p> 1X, =0. 
But, by assumption, the vectors X,, X,, ..., X, are linearly independent. 
Hence 
A, =A, =... =A, =0 


System (2. 26) has thus a unique solution, namely the zero vector and ac- 
cording to Theorem 2.4 M is therefore a regular matrix. This completes 
the proof. 

Proof of Theorem 2.6. 

1. Necessity. Let system (2.1) be solvable. This means that the vector 
Bcan be represented as a linear combination of vectors A, (the columns 


479 


of the matrix A): 
Rn 
{/=1 


Consider an arbitrary solution A=(A,, A, «.-,4,) of system (2.25): 
(A, A)=0, f= 1,2, ..., 2. (2.31) 


Scalar-multiplying both sides of the vector equality (2.30) by A and apply- 
ing (2.31), we obtain 


i 
(B, A) = 2a,(Ap A)=0. 
=1 
This proves necessity, 
2. Sufficiency. This part of the proof is somewhat more complicated 
than the preceding. According to Theorem 2.2, among the solutions of the 


homogeneous system (2.25) there are (m—r) linearly independent vectors 
(here r is the rank of the matrix A), Let these vectors be 
¥,=(%", ye, ere ay b= 1, 25.4254 m—T, 
By assumption, 
(A, ¥,)=9, f=1,2,...,m—r; fo1,2,..., 2. (2,32) 


Among the vectors A, constituting the matrix of coefficients A of sys- 
tem (2.1) there are r linearly independent vectors. To be specific, let the 
linearly independent vectors be 


We shall show that the vectors | 
As As. eer Acn. Vip Vow oes, Vacs (2.33) 


constitute a linearly independent system. Let 


m= 


2aA/t p> B,Y¥,=0. (2.34) 


Scalar-multiplying both terms of (2.34) by A,;(/=1,2,...,7r) and Y,(/=1, 2, ... 
...,m—r) and applying (2.32), we obtain 


r 
2i9,(Ap A;)=0, i=l, 2, aay ly 


m-fr 
DB Y, Y)=0, f=, 2,...,m—r. 
f=1 


According to Lemma 2.1 the matrices ||(A,, A,)||, and {I(Y;, Y)il,-- are regular. 
Hence, from Theorem 2.4 we have 


a,=a,=...=0,= 8,=$,=...=B,_,=9. 


Thus, the system of vectors (2.33) is linearly independent. The number 
of vectors in the system is r+(m—r)=m. Therefore, any vector in the m- 
dimensional space can be represented as a linear combination of these vec- 
tors. In particular, 


B=) xAj+ BY (2.35) 


Multiplying both sides of (2.35) by Y, and applying the conditions of the 
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theorem, according to which 
(B, Y,) =0, é=1,2,...,m—r, 

and relationship (2.32), we obtain 

m-? 

DIY YP=0, f=, 2, ...,m—r, (2.36) 
By virtue of Lemma 2.1 and Theorem 2.4, relationships (2.36) lead to the 
equalities 

VFM ge =u = 9. 


Therefore, (2.35) may be rewritten in the form 


‘4 
B=% x Aj 
which indicates that system (2.1) is consistent, one of its solutions being 
the vector (x,, x,, ..., ,, 0,0, ..., 0). This completes the proof. 


If the rank of matrix A is r, the general solution of system (2.25) is the 
set of all linear combinations of some (m—r) linearly independent solutions 
of this system. The conditions of orthogonality (2.24) should, therefore, 
be checked only for the (m—r) linearly independent vectors which solve (2.25). 

According to Theorem 2.4, the solution of the consistent system (1.2) is 
unique if the columns of matrix A are linearly independent. Condition 
(2.24) does not impose limitations on vector B only if r=m, Consequently, 
the criterion of solvability of system (2.1) for any free-term vector reduces 
to linear independence of the rows of matrix A, 

Hence, for the solution of system (2.1) to be unique for any vector B it 
is necessary and sufficient that A be a square regular matrix. 

The following simple proposition (Exercise 8) is also noteworthy: 

If A is a square matrix, uniqueness of solution of the homogeneous sys- 
tem corresponding to system (2.1) is a necessary and sufficient condition 
for the consistency of system (2.1) for any vector B. 

2-7. Until now we have not actually solved a system of linear equations, 
One way of solving a system of the form (2.1) follows directly from the pre- 
ceding theory of linear systems and is as foilows. 

First determine the ranks of the matrix of coefficients A and the aug- 
mented matrix A of the system. 

If these ranks are not equal, the system is unsolvable. Otherwise, we 
can determine the general solution of the system. To compute the rank of 
a matrix we must find the regular square submatrix of maximum order. 
Let it be denoted by A. The equations whose coefficients are not elements of 
matrix A arethenomitted. The unknowns whose coefficients are not elements 
of matrix A are transferred to the right-hand side. We thus obtain a sys- 
tem of equations with a square regular matrix A and a free-term vector 
which is the sum of some vector and the linear combination of several vec- 
tors. The values of the coefficients in the linear combination are arbitrary. 

Next we apply Cramer's rule, which expresses the unknowns corres- 
ponding to the columns of matrix A in terms of the remaining unknowns of 
the system. This completes the construction of the general solution of 
the system. 

The reader will no doubt see that if the parameters m and a are large 
this method of solving systems of linear equations involves computation of 
numerous high-order determinants which is most time consuming. Thus, 
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the above method for finding the general solution of a system of linear equa- 
tions is not practical, 

There are many other, simpler methods for solving systems of linear 
equations with numerical coefficients. We shall deal here with one. 

We refer, as before, to system (2.1). The method described below is 
known as the complete elimination method. This method is due 
to Jordan and Gauss, and is known also as the Gauss-Jordan reduc- 
tion method. 

The complete elimination method is based on two elementary transforma- 
tions of the augmented matrix. 

(1) A row of the augmented matrix is multiplied by a nonzero number. 

(2) A row multiplied by an arbitrary number is added to one of the rows 
of the augmented matrix. 

It can be easily verified that each elementary transformation (and conse- 
quently any sequence of elementary transformations) produces an augmented 
matrix of some new system which is equivalent to the initial system. Proof 
is left to the reader (see Exercise 9). 

The complete elimination method has a finite number of steps. In the 
first step we choose any nonzero element among the elements of the 
matrix of coefficients A (the principal part of the augmented matrix A). 
This element is called the direction [pivot]element of the given step 
(transformation). The row and the column containing the direction element 
of the transformation are, generally, called the direction row and 
column of the given transformation. 

Dividing all the elements of the direction row of the augmented matrix A 
by the direction element, we obtain a new direction row. Further, from 
each row of the matrix A (except the direction row) we subtract the new 
direction row multiplied by the element located at the intersection of the 
direction column and the row being transformed. This completes the first 
step of the complete elimination method. 

Let A’ denote the matrix obtained after this transformation from A. The 
matrix A’ was produced from the matrix A by one elementary transforma- 
tion of type (1) and (»m—1) elementary transformations of type (2). Hence, 
the system of equations with the augmented matrix A is equivalent to the 
initial system (2.1). 

Having carried out the transformations we should have zeroes in the 
direction column, except for the direction element which should be equal tol. 

All the elements in the first n columns of the matrix A’ apart from those 
in the direction row and direction column as determined in the first step 
will be called the principal part of the matrix 4’, 

The direction element of the second step is chosen from the nonzero ele- 
ments of ihe principal part of the matrix A’, The second step is carried out 
precisely as the first. Observe that the second step does not affect the 
column chosen in the first step as the direction column, since the element 
at the intersection of this column with the direction row in the second step 
is zero. In matrix A’, obtained after the second step, the direction columns 
of the first and second steps are unit vectors. The unit component of each 
of these vectors is in the corresponding direction row. Subsequent steps 
of the complete elimination method are carried out analogously. Let AM 
be the matrix obtained after k steps. To each step there corresponds a 
characteristic direction row and direction column, In matrix A there are 
k rows and & columns which have been used as direction rows and columns 
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in the preceding steps. Therefore, & columns of the matrix A® are unit 
vectors with units in the corresponding direction rows. All the elements 

in the first 2 columns of the matrix AM apart from those in the direction 
rows and columns of the preceding steps constitute the principal part of this 
matrix. The direction element of the (k+1)-th step is chosen from the non- 
zero elements of the principal part of matrix A“, Then, matrix A® is sub- 
jected to the same sequence of elementary transformations as in the first 
step resulting in the matrix A“+” with (e+1) unit columns, 

Subsequent steps are carried out as long as direction elements can be 
chosen, If after the k-th step the principal part of the matrix A™ does not 
contain any elements or comprises zeroes only, the solution process is 
terminated. 

This process, obviously, involves no more than t=min{m,n} steps, the 


total number of multiplications and divisions not exceeding me Sa 


2-8. Let the process of solution terminate after the /-th step. The matrix 
AM = (A), A, ...) An), BO) = (A, BY) 


is obtained from the matrix A as the result of elementary transformations. 
Hence, the system of linear equations 


AOX = BY (2.37) 


is equivalent to system (2.1). Since the process of solution is terminated, 
the main part of the matrix A either does not contain a single element, or 
consists of zeroes only. 

Let us first assume that among the rows of matrix A there are such 
which have not served as direction rows in any of the steps. Consider one 
of these rows, say, the i,-th row. It is easy to verify that aj}=0 for j=1, 
2,...,@. Indeed, if / is the number of the column which served as a direc- 
tion column in one of the steps, a‘)=0 because the only nonzero element of 
the /-th column is in the corresponding direction row. If, however, the 
j-th column was never chosen as a direction column, the element af’) 
is in the main part of the matrix A” which, by assumption, contains only 
zero elements. 

Thus, the i,-th equation of system (2.37) has the form 


Ox, +0x,+...+0x, = of. (2.38) 


1. If 0940, equation (2.38) is contradictory. Therefore, system (2.1), 
being equivalent to an inconsistent system (2.37), has no solutions, 

2. Ifdj?=0, the i,-th equation of system (2.37), being an identity (0= 0), 
imposes no restrictions on the unknowns and as such may be omitted. 

Examining all the rows of the matrix A which were not used as direc- 
tion rows we either establish unsolvability fo the system, or else omit all 
these rows. 

Suppose that case (a), which points to unsolvability of system (2.1), did 
not arise. Then system (2.37) is reduced to those equations which corres- 
pond to the direction rows of the matrix A. 

System (2.37) is thus reduced to precisely J equations (as the number of 
direction rows), To be specific let us assume these to be the first / equa- 
tions. We rewrite the resulting system of equations in the form 


n 
a atiz,= 00, P21, etch: (2.39) 


483 


System (2.39) is, obviously, equivalent to system (2.37) and hence to the 
initial system (2.1). In particular, / may be equal to m, This means that 
all the rows have been chosen as direction rows in the process of solution 
and, in thiscase, there is no need to omit any of equations (2.37). The ini- 
tial system is, thus, either inconsistent, or else reduces to system (2.39) 
equivalent to (2.1). We shall show that system (2.39) is consistent and that 
its general solution can be determined without any calculations. 

Without loss of generality, take the i-th direction row (i=1, 2, ...,@ to 
correspond to the i-th direction column (this can always be achieved by 
suitably renumbering the unknowns of system (2.39)). Then, following the 
complete reduction method described above 


0, t#/, 
(ty) — 
eat ix/, GS led scat: 


System (2.39), thus, may be rewritten in the form 


x= 0 — al) xp es ey aes 2 (2.40) 
=t+1 
We assume that 

X,= (00, Bi, ee ., Of", 0, 0, Ln -, 9), 

X,=(—a),—al?, ...,—af?, 0,0, ...,0, 1, 0, ..., 0), 

ee memmmememmal 
a 
s=/i+1, ...,4. 

X, obviously solves system (2.40), and the vectors X,(smi+1, ...,2) satisfy 


the homogeneous system of equations corresponding to system (2.40). 

The rank of the matrix of coefficients of system (2.40) is / (the coeffi- 
cients of the first / unknowns constitute a unit matrix of order /!), The vec- 
tors X,(s=i+1,...,2) are linearly independent (their last a—! components 
constitute a unit matrix of order na—/). Therefore, according to Theorem 
2.3, the general solution of system (2.40) has the form 


Xt | aXe (2.41) 


where @,,,, G43, -..,@, are arbitrary coefficients. 

System (2.40), however, is equivalent to system (2.1). Hence, (2.41) 
is the general solution of system (2.1). 

Each step in this method results in the elimination of some unknown 
(corresponding to the direction column) from all the equations of the sys- 
tem except the one corresponding to the direction row of the particular step. 
This explains the name of the method—complete elimination 
(reduction) method. 

Systems of linear equations are also often solved by the Gauss reduc- 
tion method which is similar to the complete elimination method. The 
difference is that in the Gauss reduction method each step results in the 
elimination of some unknown only from those equations which do not corres- 
pond to the direction rows of the preceding steps. Thus, the number of the 
transformed rows in the k-th step of the Gauss reduction method is n—&, 
whereas in the complete elimination method all the rows are transformed 
in each step. 

Comparing the amount of work involved in each method, we see that the 
complete elimination method involves more operations than the reduction 
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method. However, the complete elimination method is of special signi- 
ficance in linear programming since it provides the algorithmic basis for 
all the finite linear-programming methods. 


§ 3. Convex sets 


3-1. In 1-2 we defined the n-dimensional Euclidean space £,. Here we 
deal with various sets of points (vectors) in the space &£,. If all the points 
of set G, belong to set G, this is denoted by G,cG, (G, is contained in G,). If 
the set G, contains only one point X, and this point belongs to set G,, this 


is denoted thus: 
XE G,. 


In particular, the notation Ge&£, indicates that the set G consists of points 
of space E,. At the end of the first section we introduced some important 
sets of the space &, namely the r-dimensional subspace, the r-dimensional 
linear manifold (09<r«en). Particular cases of a linear manifold are the 
hyperplane (r=n—1) and the line (r=1). 

Consider any hyperplane with the equation 


(A, = % N,x) = C. (3.1) 


The hyperplane (3.1) divides the Euclidean space into two half-spaces 
denoted by 
(A,*X) aac, (A, X) ee. (3.2) 


Any line in the space 2&, has the equation 
X=A+Bt, —o<t<ov, (3.3) 


where A and B are some vectors in £&,. Vector 8 is generally called the 

direction vector of the given line. If the parameter ¢ in equation 

(3.3) is bounded above or below by a finite number, the set of points is cal- 

led a ray (ora half-line) with direction vector 8, Ifthe parameter ¢ is 

bounded above and below bya finite number, equation (3.3) definesa segment. 
Any point of a segment specified by the equation 


X=A+Bt, axt<B, 


can be represented as a linear combination of the end points: 


A+Ba, A-+Bf. 
Indeed 
A+ Bt=p, (A+ Ba) +p, (A + 5B), (3.4) 
M=poes Hage: (3.5) 
Note that 
B,+y,=1, p, 20, po. (3.6) 


Conversely, it can easily be shown that for any yp, and yp, satisfying (3.6), 
there exists a number t between a and 8 such that equalities (3.5) are satis- 
fied and, consequently, (3.4) applies. 
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A segment whose end points are A’ and A’ is thus a set of points X of 


the form 
X=pA'+(1—p)A", Opal. 


A sphere of radius g>0 with its center at the point A€Z£, is defined as 
the set of all points XE€2£, such that 


|X—A|<e. (3.7) 


The sphere (3.7) constitutes a e-neighborhood of the point A, If, together 
with point X, the set GEE, also contains the e-neighborhood of this point for 
some e>0, X is said to be an inside point of the set @ Anoutside 
point of the set G has a certain neighborhood which is located outside G, 

If any neighborhood of a point X contains points both belonging to G and 
points not belonging to G, then, by definition, X isa boundary point of 
the set G. A set containing all its boundary points is said'to be closed. 

It can easily be shown that any linear manifold (in particular, a hyper- 
plane and a line), a half-space, a ray, and a segment are closed sets (see 
Exercise 11), 

Furthermore, it can easily be verified that the union of a finite number 


A 
of closed sets G, G,,...,G,(U Q,) andthe intersection of any number of closed 
fut 


sets G, G,...,G, ...(N@G) are also closed sets. Proof is left to the reader 
(see Exercise 12). 

A set G is said to be bounded, if there exists a number e¢ independent 

of X such that 
|X|<e 
for all XE@. 

The famous Bolzano-Weierstrass theorem is readily extended to the a- 
dimensional Eucledian space: 

From any bounded sequence of points of space £, we 
can choose a convergent subsequence. The proof of this 
theorem is left to the reader (Exercise 13). 

Let F(X) be a function which relates to each point X, of the set GcE, a 
definite real number F(X,). The set @ is called the domain of defini- 
tion of the function F(X). 

The function F(X)is said to be continuous at the point X, in its domain of 
definition if for any e>0 there exists 8>0 such that for any point X in the 
domain of definition of F(X) separated from X, by less than 8(| X—X,|< 4) 


| F(X)—F(X,)|<e. 


A function defined and continuous at each point of set G is said to be con- 
tinuous over the set G. If F(X)is a function continuous over a bounded closed 
set G, it attains its upper and lower bounds in the set, i.e., there exists 
points X’, X°€G, such that 
F(X’) == sup F(X) = max F(X), 
xe@ XG 
F(X") = int F(X) =min F(X), 
X€G XaG 


The proof of this proposition is based on the Bolzano-Weierstrass 
theorem and is carried out precisely as for a function of one variable (see 
Exercise 14), Simple examples of continuous functions are the linear function 


n R AR 
F(X) =(C, X)= Qe,x,; and the quadratic function F(X)=X7|| ajX= 2 Dy @; ,X1% 
s=1 wi ist 


486 


Each of these functions is continuous on the entire space £, (see Exer- 
cise 15). 

3-2. A set GCE, is said to be convex if, together with any two points 
A and B, it also contains the segment joining these points: 


pA+(1—p)B, Opi. 


It can easily be verified that if @ is a convex set and points P,, P,,...,P, 
are contained in G, then for any a,50,/=1, 2,...,s satisfying the condition 


t 
2 a;=1, 


m1 
5 
the point p? a,P,E G (see Exercise 16). 
=1 


All the sets mentioned in 3-1 are convex sets (see Exercise 17). 

It can be shown that the intersection of any number of convex sets is a 
convex set (see Exercise 18). An important class of convex sets comprises 
the so-called convex cones, defined as follows. 

We shall say that a closed convex set TcE, is a convex cone with 
apex at point P,, if for any vector P€T and for any p20 the vector 


P.+p(P—P,)€ T. 


It can easily be verified that any linear manifold is a convex cone with apex 
at any point of the manifold (see Exercise 19). Another example of a con- 
vex cone is the following. Let D be an arbitrary closed convex bounded set, 
and P, some point outside D. The set of all rays issuing from P, and 
crossing D is a convex cone (see Exercise 20). 

We shall now state and prove one fundamental proposition relating to 
convex sets. This proposition is the basis of a number of important facts 
in the theory of mathematical programming. 

We first define: we shall say that a hyperplane 


(A, X)=c 


separates the sets G, and G, if (A, X)<e for all XEG, and (A, X)>c for all 
X€G,. In other words, a hyperplane separates the sets G, and G, if and only 
if G, lies in one of the half-spaces generated by the hyperplane and Q, in the 
other. 

If all the inequalities in the definition of the separating hyperplane are 
strict inequalities, we say that the given hyperplane strictly separates 
the sets G, and G,. 

A hyperplane strictly separating the sets G, and G, has, thus, the follow- 
ing properties: all the points of set G, are inside points of one of the half- 
spaces generated by the hyperplane, and all the points of set G, are inside 
points of the other half-space. 

If a set G, lies inside one of the half-spaces generated by hyperplane II, 
and @, in the other half-space, we say that the hyperplane II strictly 
isolates the set G, from the set QG,. 

Theorem 3.1. (Hyperplane separation theorem.) Let G, and G, be any 
two closed convex sets having no poinis in common, and let at least one of 
the two be bounded. With these assumptions there exists a hyperplane 
strictly separating the sets G, and @Q,. 

Proof. To be specific, let G, be the bounded set. 


487 


Let 
o = inf |P,—P, |, 
where the lower bound is taken over all the possible points P,E€ G, and P,€G,. 
By definition of the lower bound, there exist sequences {P\} and {P”}(PeG, 
P“ €@,) such that 
tim | Pw _ p® | 9, (3.8) 
-> aD 


Since G, is a bounded set, sequence {Pp} is bounded. Sequence {P{*} is also 
bounded, since 

| Ps] =| (Px? — PY) + PM? | <j PIP — PP || PY]. 
Applying the Bolzano-Weierstrass theorem, we choose from the sequence 
{Pp and {Pi} two subsequences converging to the points P} and P;, respec- 
tively. Since the sets are closed, P;€G,, P;€G, Thus, there exists a se- 
quence of superscripts k;,i=1, 2, ..., such that 


jim | PY) — pt) =0; Kim. | pho _ pS | =0, 
> >m 


Passing to the limit in| Pp“? —p*°| as i—.oo and applying (3.8), we obtain 
|P}—P,|=o= min |P,—P,}. (3.9) 
P,€Q, 
P,éG, 
By assumption, the sets G, and G, do not intersect, therefore o>0. 


Let . ; 
R,=pP; +(1—p)P;, 


where 0<p<l. Let JI, be the hyperplane defined by the equation 
(A, X)=(A, R,), (3.10) 


where A=P,—P,. Substituting we observe that II, contains the point R,. We 
shall show that the hyperplane II, strictly separates the sets G and @d,. The 
proof is carried out by reductio ad absurdum. 

Let Q€G, and also 


(A, Q)<(A, R,). (3.11) 


Since P, and Q belong to the convex set @, any point Q,=(1—e)P;+eQ, for 
O<eal, also belongs to G. 
We calculate the square of the distance between the points Q,€G, and P,€ G:: 
|Q.—P; |*=|(1—e) Pj +eQ—Pi|"=|A+e(Q—Pi) "= 
=e'(Q —Pi|*+2e(A, Q—P:)+/ Al’. 
We shall now show that 
(A, Q—Pi) <0. (3.12) 


ata (A, R)=(A, pPl+(1—p) P}) = 


=(A, (¥—1)(Pi—P3)+ Pi) = 
=(A, (u—1) A+P;)=(p—1)|Al?+(A, P3), 


but |A|/’=o0*>0, p<cl. Hence, 
(A, R,)<(A, Pi). 
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Comparing this result with (3.11), we obtain 


(A, Pi)>(A, Q), 
which is equivalent to (3.12). 
Let (A, Q—P,;)=a (a<0), |Q—P;|=f. Then 


|Q,.—P; | = Pte" + 2ea +o°, 
Taking 6>0 such that 


—@ 


rrp) <2, 
we assume 
a 
e=— ppl 


The point Q,€G,, since 0<e,<1. On the other hand 
. 8 
| Q,,— P| = 08 + 2€,0.+ esp" < 0° — 7,0 (2—0) <ot. 
Thus, having assumed (3.11), we obtain a contradiction to (3.9). Hence, 


(A, Q)>(A, R,) 
for any QEG,.. Analogously we show that 


(A, Q<(A, R) 


for all QEG,. 


The hyperplane 
(A, X)=¢, 


where A=P,—P,, c=(A, R,), R,=pPi +(1—p) Pi, O<p<1, thus strictly separates 
the sets G. and G,. This completes the proof. 

If we omit at least one of the assumptions imposed on sets G, and G, in 
Theorem 3.1, the proposition no longer applies (see Exercise 21), 

Sometimes it is not necessary that the hyperplane strictly separates two 
sets and it suffices that the hyperplane only separates these sets. In these 
cases the following proposition is useful. 

Theorem 3.2. If G, and Q, are any two nonintersecting convex sets, 
there exists a hyperplane separating G, and @,. 

Theorem 3.2 is not used in what follows and therefore the proof is not 
given here, The interested reader is referred to Exercise 23. 

3-3. As acorollary of the theorem of separating hyperplanes, we have 
two propositions which are used in several chapters. 

We shall say that the hyperplane II defined by the equation 


(A, X)=c 
is the support hyperplane of the set @ (the support of @) at the point 
Pea if 
(a) (A, P,)=c (II contains the point P,), 
(b) (A, P)<c for all PEG 
or 
(A, P)=c for PEG 


(the set G lies in one of the half-spaces generated by II). 

Corollary 3.1. (Theorem of the support hyperplane.) If P, is a boundary 
point of a closed convex set G, there exists a support hyperplane of G at 
the point P,. 
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Proof. Since P, is a boundary point of the set G, there exists a sequence 
{P,}of points outside G which converges to P,. Consider the convex sets G, 
and G,, the former being the point P,, and the latter coinciding with G. Ac- 
cording to Theorem 3.1 there exists a hyperplane II, defined by 


(A,, X) = Cy, (3.13) 

such that 
(A,, P,)> Cy, (3.14) 
(A,, P)<c, PEG. (3.15) 


Without loss of generality we may take 
|A,J/<1, |e,l<1. 


Applying the Bolzano-Weierstrass theorem, we choose from the vector 
sequence {A,} and the number sequence {c,} two subsequences converging to 
A and ¢, respectively. We shall show that the hyperplane 
(A, X)=c 
is the required support hyperplane. 
Indeed, passing in (3.15) to the limit over the subsequence of the sub- 
scripts & for a fixed PEG, we obtain 
(A, P)<c, PEG. (3.16) 
In particular, 
(A, P,)<e. (3.17) 
On the other hand, passing to the same limit in (3.14), we obtain the in- 


equality 
(A, P,) 2c. (3.18) 


Comparing (3.17) and (3.18) we obtain 
(A, P,)=c. (3.19) 


Relationships (3.16) and (3.19) show that the hyperplane (A, X)=c is the 
support hyperplane of G at the point P,. This completes the proof. 

The conditions of the theorem of the support hyperplane can be weaker: 
it is not necessary to assume that the set G is convex (see Exercise 22). 

We define the distance e(P, G) between the point P and the set G as 


o(P, G)= inf |P—Q|. 
Qea 
Let G, and G, be closed disjoint sets, at least one of which is bounded. 
A point Ph, €G, is said to be the point of G, closest to the set G, if 
e(P:, G.)= int e(P,, G). 
P,€G, 


The existence of this point was established in the proof of Theorem 3,1, 


Corollary 3.2. Let G, and G, be sets satisfying the conditions of Theo- 
rem 3.1 and P; be the point of G, closest to G. There exists a hyperplane 
Il defined by the equation 


(A, X)=e, 
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such that 

a) (A, P)<e for PEG, 

b) (A, P')=c, 

c) (A, P)>e for PEG, 

(Hyperplane II is the support of G, at the point P* and strictly isolates the 
set G, from the set G,.) 

Proof. We introduce a hyperplane 


(A, X) = (A, R,), 
where 
A=(P\—P'), R, = pP) +(1—p) Pi. 
Here P* is the point of G, closest to G,. 
In the proof of Theorem 3.1 we established that for 0<yp<! 


smin(A, P,)>(A, R,) (3.20) 
max (A, P,)<(A, R,). (3.21) 


Letting the parameter p in (3.21) approach zero, we obtain 
max(A, P,)<(A, P%). (3.22) 
P,eG, 


We observe that (A, R,)=p|Al|*+]A, P{|\does not increase as p decreases. 
Therefore, when the parameter p in (3.20) approaches zero, the inequality 
remains Strict: 

mmin(A, P\)>(A, PI) (3.23) 


It follows from (3.22) and (3.23) that the hyperplane II 
(A, X)=(A, P%) 


has the properties required and is, thus, the desired hyperplane. 

3-4, Consider a three-dimensional space. Among the convex sets in 
this space we have one-dimensional (segment), two-dimensional (circle), 
and three-dimensional (sphere) figures. A characteristic property of a 
three-dimensional convex figure is that there is no plane which can contain 
it. A two-dimensional convex set can lie in some plane; however, no line 
contains this set. A one-dimensional convex set always belongs to some line. 

These considerations lead to the following definition of a convex set in 
an n-dimensional space, 

We shall say that a convex set has e dimensions if it is con- 
tained in some e-dimensional linear manifold and cannot belong to a linear 
manifold of dimension less than ag. 

Recalling the two different definitions of a linear manifold, the dimen- 
sionality of a convex set can be defined in two equivalent forms: 

1. The dimensionality of a convex set Gc EB, is g=a—r, where r is the 
maximum number of linearly independent hyperplanes whose intersection 
contains @. 

2. The dimensionality of a convex set @ is equal to the minimum num- 
ber @ of linearly independent vectors X,, X,,...,X, such that @ is contained 


in the set of all points 
X=X, +3 a,X,, 


where X,E€G, anda,, a,,...,a@, are arbitrary numbers. 
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We now prove a useful proposition which elucidates the concept of dimen- 
Sionality of a convex set. 

Theorem 3.3. A convex set G has a dimensionality e=1 if and only if 

(a) G is contained in some e-dimensional linear manifold R; 

(b) there exist a point PEG and a number e>0 such that the e-neighbor- 
hood of the point P the set G and the manifold R coincide. 

Proof. 1. Necessity. Let G be e-dimensional and let R be a e-dimen- 
sional manifold containing G. Consider any point X,€G. Choose the maxi- 
mum number of linearly independent vectors X,, X,,...,X, among the vectors 
of the form X—X,, where XE€G. Their number s is, obviously, equal to ga. 

Indeed, the inequality s>e@ contradicts the fact that the set X—X,, XEG 
is contained in the e-dimensional subspace X—X,, XER. 

On the other hand, the set G belongs to the manifold 


s 
Xi+ Bax, 
so that s>¢aQ. 
Hence s<g. 
By virtue of the convexity of the set G 


a,X,+ $a(X,+X,)€ 0 
for any a,>0, i/=0, 1,...,@ whose sum is equal to unity. Hence, 
X,+ 3a,X,€0, (3.24) 
if 
a, 0, iml, 2,...,Q yaa. 


Let : 

Maks UX (X,€ Q). 
From (3.24) 

X £5, X,€ 0, i=1, 2,...,0 
Hence, 

X + Sav’ X,— Sai X,€ 0, 
if 


qQ Q 
7 1 
a; 0, a, 590, fm, 2,2 Da + 2%) <5: 
om eS | 


or, equivalently, 


X+ aX €0, (3.25) 

if 
x ! 

l[a,| <5. 
bay 29 

By assumption, the set X—X,, X€0O is contained in the subspace X—X), 
X€R. Hence, the linearly independent vectors X,, X,,...,X, belong to this 
subspace and the e-dimensional linear manifold coincides with the set of 
points of the form 


Xy+ yar, (3.26) 
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where a,, a,,...,a, are arbitrary numbers. Let 


y= int} > aX, |= min / & Dae % Xj); 
=1 =1y=1 


where the lower bound is taken over all the possible coefficients a, satis- 
fying 
Dlel=1, (3.27) 


The function | 3 a,X,] is defined and continuous at any point 
é=1 
a=(a,, a,,.. » GEL. 


The set of all points a=(a,, a,,...,a,) satisfying (3.27) is a closed bounded set. 
Hence, there exist numbers @,, @,,...,, such that 


B=1 XI, yaa. 


By definition, p>0, If p=0, then 


=1 


and from the linear independence of the vectors X,, X,,...,X>p, 


@=G@=... =G,=0, 


which contradicts (3.27). Hence p>0. 
Let XER and 


|X—X/|< 95. (3.28) 


According to (3.26) 


Let 


Hence, applying (3.28), we have 


Q Q 
A=Slal<2| ax] <x. (3.29) 
i=) i=) 
From (3.29) and (3.25) we have 
X=X,+ YaX€0, 


Thus, any point X of the e-dimensional linear manifold R satisfying in- 
equality (3.28) belongs to G, and GcR. Necessity is thus proved (P= X,, e=f). 

2. Sufficiency. Let R be ag-dimensional linear manifold containing G, 
and let the point PE@ satisfy (b). 

The manifold R consists of points of the form 


P+ yar, 
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where X,, X,,...,X, isa linearly independent system of vectors, According to 
condition (b) of the theorem, 

P+eX,€G 
for ix1, 2,...,0. Let R’ be an arbitrary linear manifold containing G, In 
this case the points eX;(i=1,2, ...,e) belong to the subspace X—P, XER’. 
Hence the linear manifold R’ cannot have less than e dimensions. 

The dimensionality of R does not exceed the dimensionality of any linear 
manifold containing G. This means that G is e-dimensional. Thus, suffi- 
ciency is proved. 

From this theorem it follows, in particular, that convex sets GEE, are 
n-dimensional if and only if they contain inside points. 

A zero-dimensional convex set is a point (the intersection of a linearly 
independent hyperplanes). 

A one-dimensional convex set lies on some line. Hence, assuming the 
set is closed, it coincides with either the line, a ray, or a segment (see 
Exercise 24). 

We emphasize that the definition of dimensionality introduced here refers 
only to convex sets. Application of this definition to an arbitrary set will 
not be successful since according to this definition a circle (which is ob- 
viously a one-dimensional set) is two dimensional. 

3-5. A point P of set G is called an extreme point if it cannot be ex- 
pressed in terms of two other distinct points P, and P, in G as 


P=pP,+(1—p)P,, 
where 


O<cp<cl. 


Thus, if P is an extreme point of set @ and S is an arbitrary segment in 
G, P either does not belong to S or is one of its end points. 

Examples of extreme points in linear and plane sets are the end points 
of a segment, the vertices of a polygonal, the points of the boundary of a 
circle, etc. Convex sets may have a finite (polygonal) or an infinite (circle) 
number of extreme points. Some convex sets (e.g., a line) do not contain 
an extreme point at all. We shall see below that such sets should either 
be unbounded, or open. 

We shall say that the set G is a convex hull of the set G, if it is the 


set of points of the form 
N 


a biP ‘D 


N 

where eis p;2O(¢=1, 2,...,N) and P, P,...,Py is an arbitrary finite sys- 
=1 

tem of points in G,. 


N 
The vector uP is called a convex combination of the vectors 
=t : 


P,, Py, ..-» Pw. It can easily be shown that the convex hull of any set isa 
convex set (see Exercise 25). On the other hand, any convex set containing 
G, necessarily contains its convex hull @ (see Exercise 16). 

The convex hull of a set G is, therefore, the smallest convex set con- 
taining @. We shall formulate and prove a proposition which elucidates the 
structure of a convex closed bounded set. 

Theorem 3.4. (Representation theorem.) Let G bea closed bounded 
convex setand @ the set of all extreme points of G. In this case G is the 
convex hull of the set @. 
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Proof. Proof is by induction on the dimensionality a of the Euclidean 
space £&, containing G. 

If the bounded closed convex set Ge£,, it is obviously either a point ora 
segment. In the first case, the sets G and G@ coincide, In the second case 
G* comprises the end points of the segment G, and any segment is a convex 
hull of its end points. We have thus proved the theorem for a1. 

Assume that the proposition is true for the space £,_,. We shall show 
that it also applies for B,. Let the set GCE, and P,=(p, p®,...,p) bea 
point in @. 

1. First let P, be a boundary point of the set G. Then, from Corollary 
3.1, we may draw through point P, a hyperplane II with the equation 


(A, X)=c, (3.30) 


which would be the support hyperplane of G, i.e., such that (A, X)<c for 
X€G, 

Let G, be the intersection of G with the hyperplane II. Obviously, @, 
is aclosed boundedconvex set; P,€G. To be specific we assume that the last 
component 4, of the direction vector A=(A,, A,,...,4,) of the hyperplane II is 
not zero. 

According to (3.30), for any point P=(p,, p,,...,p,) ETI 


1 nei 
= YSheitz- (3.31) 


é=1 


Consider the set G,C £,_, comprising the points P=(p,, p,,...,p,-,)€E,-, such 
that the corresponding points P=(p,, p,,.-+: Pao. Pa), Where p, is determined 
from (3.31), belong to G. 


Obviously, G, is a bounded closed convex set (G, satisfies these condi- 


tions); the point P,=(p!”, p,...,p%,)EeG. Hence, by the assumption of induc- 
tion, there exist extreme points of the set G, 


such that P, = (p\, po, oe eer ’ i=1, 2; ees N, 


= — 
Py= 2ibiPi (3.32) 


N 
biel, yp, = 0, i=1, 2, scecepas 
=1 
We define the points 
P= (p®, po, s pl , pe), f=1, 2, ve oy N, 


taking 
po=— De w+e. (3.33) 
kar" i) 


P.€G, since by assumption P,€@, (i=1, 2, ..., NM). Since the point P, satisfies 
equality (3.31), and representation (3.32) and relationship (3.33) apply, we 
have 


p= Me ppt = — "Sh Suey+e = 
° k=1 An An k=1 hn i=1 An 


n—1 


N N 
= Yu. [—% Pol +e] = Bee ee 
{=1 PPS eee in =1 
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Applying (3.32) and (3.34), we obtain 


Po= De iPi (3.35) 
where 
N 
De=ls peo, C1 2 y_candg AN: 
i=) 
We shall show that all the points P, in (3.35) are extreme points of the 
set G. Assuming the contrary for some i(l<—i<N), we have 


P;=pQ, +(1—p) Q,, O<p<l, Q,#Q,; 
Q,= (9, 6,-0.6, MEG, s=t, 2. 


Since II is a support hyperplane of the set G, we have 
(A, Q)<c, s=1, 2. (3.36) 


Further, 
(A, P)=p(A, Q)+(—p)(A, Q,) =e. 


Hence, applying (3.36) and the condition 0<p<1, we obtain 
(A, Q,)=(A, Q,) =c. 
Therefore Q.€G,, s=1,2 and consequently 


QO, eee GeuEG, s=l, 2. 
Thus, 
P;=pQ,+(1—4)Q, Op, 
Q,€G, s=1, 2. 

Since P; is an extreme point of the set G, we have Q,=Q,=P;, whence, ap- 
plying to relationship (3.31), we have Q,=Q,. This contradiction shows that 
P, is an extreme point of the set @. 

2. Now let the point P, be an inside point of the set G@. Consider a line L 

X=P,+Rt, —wo<t<oo, (3.37) 


where R is an arbitrary nonzero vector. The intersection of G and L is, 


obviously, the segment L with 


X=Pi+Rt, t.atat,. 
Let 


X,=P,+Rt,, X,=P,+Rt,. 


Then 
Pr=pX,+(1—p)X, O<p<l. 


Obviously, X, and X, are boundary points of G. They can, therefore, be 
represented in the form (3.35): 


N 
x= Supe 
f=1 


Pane 
Dera, wi)>0, d= 1, 2, ..., My. 
=1 


Pp) are extreme points of G, s=1,2. Hence, 


N, Ny 
Poa 2 ES RRP ED 
=i =1 
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Let 
PY), d= 1, 2, 2... Ny 
P,={ Py, £=N,+1, N42, ...,M, +N=N, 
J bey, =1, 2, ..., M, 
B= pay, IN +1 N +2 005M. 


Then ‘% 
Po= Dep (3.38) 


Thus, according to (3.35) and (3.38), an arbitrary point of the set G can be 
represented as a convex combination of a finite number of extreme points 
of this set. And, conversely, any convex combination of a finite number 
of points of a convex set belongs to this set. G is therefore the convex hull 
of G*, the set of its extreme points. 

Assuming that the theorem for an (a—t)-dimensional space is valid, itis 
true also for E, and, as has been shown previously, the theorem applies for 
n=1. This completes the proof. 

Let G be a convex set. The set G, is said to be the skeleton of @ if 

(a) Gis a convex hull of G,, 

(b) any subset of G, does not satisfy (a). Figuratively speaking, a ske- 
leton of a set is the smallest set of all those satisfying (a). 

We observe that not every convex set hasaskeleton, Forexample, an open 
interval does not have a skeleton. The representation theorem establishes 
the existence of askeleton for any closed bounded convex set. It is found thai 
the skeleton of set G satisfying these requirements coincides with G*, the 
set of extreme points of G. 

In conclusion, we give one obvious corollary of Theorem 3. 4. 

Corollary 3.3. Any nonempty bounded closed convex set contains atleast 
one extreme point. 


EXERCISES 


1. Applying the properties of scalar multiplication, prove the triangle inequality: 
|A—B| <|A—C}+|C—B|, 


where A, B, C are any points in E,. 
2, Prove the equality 
aee i all cy Moll aryl nH ll nln 


where the matrix |} ay|| , (the inverse of || Giz||_n) is defined by (1.7). 

8, Prove that the inverse matrix for any regular matrix || a@;7||, is unique. 

4, Verify that a singular matrix does not have an inverse. 

5. lf a vector can be written as a linear combination of linearly independent vectors, this representation 
is unique. Prove. 

6. Using the criterion of linear independence of a system of vectors, prove that the rank of any matrix 
is equal to the maximum number of linearly independent rows (columns) of the matrix. 

1, Prove the equivalence of the following two definitions of an r-dimensional subspace: 

(a) an r-dimensional subspace is the set of all linear combinations of r linearly independent vectors; 

(b) an r-dimensional subspace is a set which, together with any two vectors, contains any linear combi- 
nation of these vectors, the maximum number of linearly independent vectors inthe set being r. 
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8, If A is a square matrix, solvability of the system of equations 
AX = B 


for any vector B is a necessary and sufficient condition for uniqueness of solution of the corresponding homo- 
geneous system 
AX =0. 


Prove. 

9, Verify that the elementary transformations of the augmented matrix of a system of linear equations 
introduced in 2-7 do not affect the general solution of the system. 

10. Applying the complete elimination method determine the general solution of the following systems: 


(a) 2x,+3x,+ x,—2x, =i, 
2x, + x3+3%,+ %—3x,=3, 
X, + 2v,+3x,— 3x, + 2x, =2 2; 


(b) — *, -+-3x, + 2x,—3x, + 2x, +3x,=3, 


2x, —3xg— 2X4 +3%,—2x,— x =l, 
3x, —3x, +3x,—2z, + x, = 2; 


(c) x, +2x,+3x,=1, 
—2x,-+ %y— %=2, 
— £,+3x,+2x,=3, 


11, Verify that any linear manifold, space, ray, and segment are closed sets. 

12, Prove that the union of a finite number of closed sets and the intersection of any number of closed 
sets are closed sets. 

13, Prove that for any bounded sequence of points in the space E, there exists a convergent subsequence. 

14, A function defined and continuous on a bounded closed set of space E, attains its upper and lower 
bounds on this set. Prove. 

15, Show that the linear function 


a 
F (x)=(C, X)= Dor 
é=1 
and the quadratic function 


ce | i | 
F(X)=XT ay linX= DY Daisxxy 
j=siies 
are continuous at each point of the space E,,. 
16. If G is a convex set, then from 


5 
P,EG, a0, t=1,2,....8 Da=l 
{s1 
it follows that 


$s 
P= D)a:P;€4. 


i=i 
Prove, 


17. Show that a linear manifold, a half-space, a ray, a segment, and a sphere are convex sets. 
18, The intersection of any number of convex sets is a convex set. Prove. 
19, Verify that any linear manifold may be considered as a cone with its apex at any point of the manifold. 
20, Let D be a bounded closed convex set in the space E,, and let P, @ D. Show that the set of all 
rays issuing from P, and crossing D is a convex cone. 
21, Show that omission of any of the assumptions of Theorem 3.1 with regard to the sets G, and G, 
renders this proposition, generally speaking, invalid. 
22, Let G be a convex set in the space E,, and let P € E, not be an inside point of G. Applying 
Theorem 3.1 and Corollary 3.1, establish the existence of a hyperplane passing through P and containing G 
in one of its half-spaces. 
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23. Prove Theorem 3.2. Hint: consider a set G comprising the points 
X=X,—X,, 


where X;€G;, ¢==1, 2 and apply the proposition of the preceding exercise to this set at P=0. 


24, Show that a one-dimensional closed convex set coincides either with a line, or with a ray, or with 
a segment. 


25, Prove that the convex hill of any set is a convex set. 

26. Improve the proof of the representation theorem by showing that any point of a bounded closed con- 
vex set is a convex combination of no more than a-+1 extreme points of this set. Hint: let the line L with 
equation (3.37) pass through P, and some extreme point of the set G. 

27, Show that an open interval has no skeleton. 
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SUBJECT INDEX 


All-integer programming, 5, 18 


Basis, artificial, 164 

— components, 253 

—— of a program, 66 

—, dual, 252 

— of a system of vector, 471 

— of quasiprogram, 366 

— of support program, 66, 122, 
252, 429 

— production modes, 457 

Block programming, 5, 18 


Canonical form, 20 


Classification of finite methods, 452 


Cofactor of matrix element, 470 
Column, direction, 177, 197 


Combination of vectors, convex, 494 


—-—, linear, 470 
—-—, linearly independent, 471 


Components, basis, of a program, 66, 122 


— of a vector, 463 

Cone, convex, 487 

—, —, polyhedral, 51 

Consistency of a system of linear 
equations, 473, 474, 478 

Constraint of a polyhedral set, 
rigid, 35 

Continuity of a function at a 
point, 486 

Convex programming, 4 

——, quadratic, 4 

Criteria, optimality 106, 109 

—, unsolvability, 219, 230 

Cycling, 238 


Decision multipliers, 75, 105 

— vector, 105, 109 

Degeneracy, 151, 281 

Degree of degeneracy of a support 
program, 241 

Determinant of a matrix, 468 

Dimensionality of convex set, 491 

~ of vector space, 471 

Direction vector of the edge, 41 


Distance between points, 465 

Domain of definition of a function, 486 
—-— of aproblem, 7, 58 

Dual (conjugate) pair of problems, 75, 81 
— simplex method, 18, 251 


Edge, bounded, 41 

— of a polyhedral set, 38, 39 ff, 

—, unbounded, 41 

Element, direction, of a step (transformation) 
177, 197, 482 

—, matrix, 466 

Elementary-transformation parameter, 148 

Equality of vectors, 463 

Equation of hyperplane, 478 

Equivalence of systems of linear equations, 
475 

Evaluations of augmented restraint vectors 127 

— of problem restraints, 127 

—, relative, 149 

Extrabasis variables, 122, 144, 253 


Faces, corresponding, of a set, 48 
—,q-dimensional, 38 

Finite methods, classification of, 452 
Form of a problem, canonical, 19 


Geometrical interpretation of dual problem, 78ff. 


— — of problems, first, 22, 70 ff. 
——-—, second, 25, 71 ff. 


Half-space, lower, 70, 135, 485 
—, upper, 70 

Hull of a set, convex, 494 
Hyperplane, 472 

—, linear-form, 70 

—, separating, 487 

—, strictly separating, 487 

—, support, 489 


Inequality, triangle. 465 

Integer programming, 5, 18 

Interpretation, geometrical, of dual 
simplex method, 259 

—, —, of Hungarian method, 376 

—, —, of problems, first, 22 
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Interpretation, geometrical, of second, 25 Method of inverse-matrix, 193 
—, —, Of simplex method, 134 


Inversion, 468 Method, inverse- matrix, 193, 195 
Iteration, 208 —, iterative, 18 

—, large, 447 eon —, Motzkin relaxation, 18 

—, simplex, 130 —, Neumann's, 18 

—, small, 447 — of bilateral evaluations, 411 


— of decision multipliers, 17 
— of differential equations, 18 
— of leading variables, 18 

—, potential, 18 


Kernel of a program, 273 
Key element, 177 


Length of vector, 465 


Line, 472 —, simplex. 18, 121, 352, 460 
Linear combination of vectors, 470 —, —, coordinate form, 190 
— -form hyperplane of the problem, 70 — —, dual, 251 
— form of problem, 7 —, —, first algorithm, 174 
— programming, vii, 1, 4, 6, 1% —, —, geometrical interpretation, 134 
M-problem, 162 —, —, second algorithm, 193, 200 
Manifold, linear, 472, 478 —, trench, of Gel'fand, 19 
Mathematical programming, vii, 1, 2, Methods, finite, 428 

526 —, —, Classification of, 452 
—— under total information, 4 —. —, modification of, 444 
Matrix, 466 Minor, 471 
— inverse. 470 Modification of finite methods, 444 
— Leontieff. 30 Multi-component equipment problem, 11, 16 
——-—-—, augmented, 473 —, Lagrange, 110, lle 
—, reduced, 346 Neighborhood of a point, 486 


—, regular, 470 
—, singular, 470 
—, transposed, 468 


Nonlinear programming, 4 
Norm of a vector, 465 


Method, Brown’s, 18 Optimal program, 8 

—, combined, 359 Optimality, see Criteria, optimality, and 
—, complete elimination, 482, 484 Tests, optimality 

—, Dantzig, 121 Orthogonality of vectors, 465 


—, double description, 18 Pair, dual (conjugate), of problems, 75, 
—, dual simplex, 18, 251, 352, 460 81, 100 


—,——, first algorithm, 289 


Z Parametric programming, 5 
—,——, geometrical interpretation, 259 


Paying production methods, 456 


—, —-—, second algorithm 308 Permutation, 468 

—, Gauss reduction, 484 Pivot element, 177, 482 
—, gradient, 327 Point, 464 

—, Hungarian, 17, 363, 460 —, boundary, of a set, 486 
—, —, algorithm, 390 —, inside, of a set, 486 
—, —, geometrical interpretation, 376 — of acone, 53 


— — revised, 410 ff. —, outside, of aset, 486 
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Point, saddle, of a function, 112 
Polyhedron, convex, 39 

—, restraint, 30 

Primal-dual algorithm, 390 ff. 
Problem, allocation, 10, 11 

—, augmented, 345, 364, 383 


—, auxiliary, 364, 383, 446 

—, degenerate, 68 

—, dual (conjugate), 67, 149, 203 

—, nondegenerate, 149 

Problem of optimal] weapon system, 
11, 15 

—, primal, 76 

—, transportation, 11, 14 

Problems, dynamic, 4 

— in arbitrary form, 428 

—, planning and control, 22 

—, production-scheduling, 11 

—, Static, 4 

— with homogeneous restraints, 80 

— with mixed restraints, 100 

Product of matrices, 467 

— of matrix by a scalar, 467-8 

— of vector and scalar, 464 

—, scalar, of vectors, 465 

Program, 29 

—, degenerate, 67 

—, feasible, 8 

—, nondegenerate, 67 

—, optimal, 8, 29 

—, support, 66, 122, 144, 252, 429 

—, —, degenerate, 68, 69 

-~ —, initial, 159, 230, 326 


~—, —, nondegenerate, 67,69,149, 203,429 


—, —, theorem of existence of, 165 
Programming, all-integer, 5, 18 
—, block, 5, 18 

—, linear, vii, 1, 4, 6, 17 

—, mathematical, vii, 1, 2, 5-6 
—, nonlinear, 4 

—, —, convex, 4 

—, —, Cuadratic, 4 

—, parametric, 9 

—, stochastic, 4, 18 
Pseudoprogram, 203 

Pseudovei:ex of the restraint set, 261 


Quadratic programming, 4 
Quasiprogram, 365, 384 


Rank of a matrix, 471 
~— of a system of linear constraints, 36 


—— of vectors, 471 
Ray, 485 


Representation of convex polyhedral 
set, 42 

Residue of program, 413 

Residue of quasiprogram, 366, 384 

Residues, 363 

Restraint, dual, 94 

—, fixed, 94 

—, free, 94 

— matrix of problem, 7 

Restraints, bilateral, 216, 271, 382 

—, column, 98, 103 

~—, homogeneous, 80 

—, row, 98, 103 

Row, direction, 177, 197 

Rule, Cramer's, 474 


Saddle point of a function, 112 
Segment, 485 

Set, closed, 486 

—, convex, 487 

—, —, dimensionality of, 491 

—, —, polyhedral, 34, 35, 51, 57, 58 
— of problem restraints, 58 

Simplex method, 18, 121, 352, 460 
Solution, 8 

—~ of a system of linear equations, 473 
———-—, general 476 

—, support, 61 

—, —, theorem of existence of, 166 
Solvability of a problem, 9 

— of a system of linear equations, 473 
— theorem, 166 

Space, Euclidean, 485 

—, vector, 464 

Sphere, 486 

Stochastic programming, 4, 18 
Submatrix, 471 

Subspace, 472 

Sum of matrices, 467 

— of vectors, 464 
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Support program, 909, 66 Unsolvability of a system of linear 
— solution, 61 equations, 473 
System of linear algebraic equations, 472 Unsolvability test, 368, 388 


———-—, consistent, 473 Variable, basis, 144 


————, homogeneous, 475 Variables, extrabasis, 122, 144 


—, restraint, of a set, 4d 


Vector, 464 
Test, unsolvability, 388 —, constraint, 8, 144 
Tests, optimality, 122, 123, 124,145, 146 —, cost, 82, 83 
274, 384. 413 —, decision, 105, 109 
Theorem, Bolzano- Weierstrass, 486 —, demand, 29 
Theorem, duality, first, 92, 166 —, direction, of a hyperplane, 485 
~—, hyperplane, separation, 487 —, —, of an edge. 41 
— of existence of a support program, —, —, of a line, 485 
165 —, n-dimensional, 463 
— of support hyperplane, 489 —, residue, 366 
—, representation, 494 —, restraint, 7 
—, solvability, 166 —, —, augmented, 26 
Transformation of a program, —, —, free, 115 
elementary, 126, 256, 367 —, supply, 29 
——, —, associated with the —, zero, 464 
vector, 275 Vectors, tentative cost, of production 
Transportation problem, 14 factors, 82 
Transposition of matrices, 468 Vertices, adjoining, 67 
—ofaset, 38 
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